Lesson 4 - Introduction to Python.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Programming\n",
    "\n",
    "Programs are sets of instructions intended to achieve a particular task.\n",
    "\n",
    "Programs generally try to \"abstract\" a problem so that the same program can be used to achieve many similar, but non-identical tasks.  They do this by using *variables*.  For example:\n",
    "\n",
    "     Say Hello to Mark\n",
    "     \n",
    "This small \"program\" is a concrete solution to the specific problem of saying Hello to Mark.  It solves that problem perfectly!  But I have to write many many programs if I want to say Hello to each of you in this class...\n",
    "\n",
    "     Say Hello to Pedro\n",
    "     Say Hello to Julio\n",
    "     Say Hello to Maria\n",
    "\n",
    "This is not efficient.  So we make an \"abstrat\" version of the same program:\n",
    "\n",
    "     Say Hello to {X}\n",
    "     \n",
    "Now this single program can say Hello to whomever {X} is.  Or better:\n",
    "\n",
    "     Say {S} to {X}\n",
    "\n",
    "That is a program can say anything to anybody!  \n",
    "\n",
    "...and that's all you really need to understand about programming, at this level.  It will take some practice before you become good at abstraction - it's a different way of thinking!  You will use abstraction to automate repetitive tasks, to create data models for a wide range of data structures/content, and to create \"generic\" solutions to a variety of other problems.\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The Python Programming Language\n",
    "\n",
    "**Python** is an *interpreted*, *general purpose*, *object-oriented* language that can be run on any operating system. \n",
    "\n",
    "It is pre-installed on most Linux distributions (including this VM)\n",
    "\n",
    "In the next 60 seconds you will create your first two Python programs!\n",
    "\n",
    "1. Open a terminal window.\n",
    "2. Type \"python\"  \n",
    "3. At the <code>>>></code> prompt type\n",
    "<code>\n",
    "           print(\"this is my first Python program\")\n",
    "</code>\n",
    "\n",
    "Done!\n",
    "\n",
    "Now for something a little bit more interesting:\n",
    "\n",
    "1. At the <code>>>></code> prompt type\n",
    "<code>\n",
    "           i_am_smart=True\n",
    "           if i_am_smart:\n",
    "               print(\"I am going to do well in this course!\")\n",
    "</code>\n",
    "    \n",
    "(note that the second line has a few spaces at the beginning.  This is because emptiness is meaningful in Python! You will hear me complain about this often during this course!  In my other course - Programming Challenges - you will use a much more aethetically beautiful and elegant language... Ruby!)\n",
    "\n",
    "(now press *ENTER* twice)\n",
    "    \n",
    "### Try changing the value of \"i_am_smart\" to 'False' - what happens?\n",
    "\n",
    "\n",
    "Now that you have written two simple Python programs, we will talk more deeply about what programming is, and how to write more complex apps.\n",
    "\n",
    "\n",
    "### TO EXIT FROM THE PYTHON TERMINAL\n",
    "\n",
    "type <code>exit()</code>\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Types and Structures\n",
    "\n",
    "Programming requires you to think very carefully about the KIND and STRUCTURE of data.  For example, the digit 9 and the character \"9\" are *not* the same thing (though in some very flexible languages like Perl, it interprets them in whatever way is most appropriate).  For example, hypothetically:\n",
    "\n",
    "     9 + 9 = 18\n",
    "     \"9\" + \"9\" = \"99\"\n",
    "\n",
    "Most languages (like SQL!) support a wide range of data types - integers, floating-point numbers, \"strings\", date/time, exponentials, 'booleans' - and you should select the correct one for the data you are trying to represent.\n",
    "\n",
    "Data *structures* are slightly different from data *types*.  For example:\n",
    "\n",
    "* There is data that is \"scalar\" - it has a single value (like {X} in the example above).  \n",
    "* There are also lists (1,3,5,7,9) \n",
    "* and 'Associative lists' (Chapter1 : Introduction, Chapter2 : Installing,  Chapter3 : Running)\n",
    "\n",
    "There are more complex data structures, and you will learn them later in this course.  For the moment, we will only look at simple examples that combine these simple data types, with simple data structues.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Code Documentation\n",
    "\n",
    "_**I promise you this:  when you write a Python program today, you will not remember how it works next year!  And nobody who wants to use it after you will understand what it does, or how to use it, so they will have to write another program themselves.**_\n",
    "\n",
    "This is a waste of everyone's time, and a waste of your supervisor's money!!  \n",
    "\n",
    "For this reason, the first thing I am going to tell you about is **documentation**\n",
    "\n",
    "Simple documentation in Python is achieved with <code>'# comments'</code>.  the <code>#</code> symbol tells Python that the rest of the line is NOT a part of the program, but is rather a \"chat\" with whoever is reading the code, to help them understand what the code does.  <code>#</code> can appear by itself, or after other Python commands.\n",
    "\n",
    "For example\n",
    "\n",
    "<code>\n",
    "    \n",
    "    # the next few lines set-up the initial connection to the database\n",
    "    # starting from the first record of this year\n",
    "    \n",
    "    a=14435  # set a to the first database record of the current year\n",
    "    if (db.retrieve(a)):   # check that the record exists\n",
    "        print(\"record exists\")  # inform the user that the record exists\n",
    "</code>    \n",
    "\n",
    "\n",
    "The comments allow us to \"read\" what the intent of each line is, so that we don't have to *interpret* the code.\n",
    "\n",
    "I want you to use comments, even for simple programs, because it is a **very important good habit (buen hábito).**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>\n",
    "\n",
    "</pre>\n",
    "# Reserved Words\n",
    "\n",
    "There are words in Python that are \"reserved\" because they have a special meaning in the language.  This means that you cannot use them for variable names, or the names of functions inside of your app.   These are\n",
    "\n",
    "\n",
    "| Reserved Words |  - | -  | -  | -  |  \n",
    "| ---    | --- | --- | --- | --- |\n",
    "| False |\tclass |\tfinally  |\tis |\treturn |\n",
    "| None  |\tcontinue |\tfor |\tlambda |\ttry |\n",
    "| True |\tdef \t| from |\tnonlocal |\twhile  |\n",
    "| and |\tdel |\tglobal |\tnot |\twith  |\n",
    "| as | \telif |\tif |\tor |\tyield  |\n",
    "| assert |\telse |\timport | \tpass |   |\n",
    "| break |\texcept |\tin |\traise |    |\n",
    " \n",
    " \n",
    " \n",
    "Note that the reserved words \"True\", \"False\" and \"None\" are capitalized.  All the rest are lower-case.\n",
    " \n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>\n",
    "\n",
    "\n",
    "</pre>\n",
    "# Data Types\n",
    "\n",
    "Data Types means the \"nature\" of the data that Python can represent.  Generally speaking, it can represent all of the \"core\" data types such as Numbers and words/letters/strings, and there are some other datatypes that are useful when coding.\n",
    "\n",
    "## Numbers\n",
    "\n",
    "Numbers are represented as follows:\n",
    "<code>\n",
    "123\n",
    "1234\n",
    "1_234\n",
    "1_234.56\n",
    "1.234e-56\n",
    "**0x**ffff  # hexadecimal\n",
    "**0b**010101 # binary\n",
    "**0o**377 # octal  (zero + 'oh')\n",
    "</code>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(123)\n",
    "print(1234)\n",
    "print(1_234)\n",
    "print(1_234.56)\n",
    "print(1.234e-56)\n",
    "print(0xffff)     # hexadecimal\n",
    "print(0b010101)   # binary\n",
    "print(0o11)       # octal  (zero + 'oh')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Strings\n",
    "In most cases, Strings are enclosed in single (') or double (\") quotes.\n",
    "\n",
    "For example:\n",
    "\n",
    "print(\"hello Python programmer!\")\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"hello Python programmer!\")\n",
    "print(\"hello #1 Python programmer!\")   # digits can be part of a string\n",
    "print(\"123\")   # digits can be the entire string\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>\n",
    "\n",
    "</pre>\n",
    "There are some special characters in Strings.  A few common examples are:\n",
    "\n",
    "    \\n = 'newline'\n",
    "    \\r = 'return' (important for MS Windows!)\n",
    "    \\t = 'tab'\n",
    "\n",
    "So for example:\n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"this is my \\n\\t\\t first \\t Python \\t app\")\n",
    "print(\"this is my \\n\\t\\t first \\t Python \\t app\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or you can create multi-line strings using triple-quotes:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"\"\"This\n",
    "is\n",
    "a multi-\n",
    "line \n",
    "string\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"this is \"  +  \" a way to add strings together\")\n",
    "print(\" say this twice\" * 2) # '*' is the multiplication operator\n",
    "print(\"\")\n",
    "# you can also take substrings of strings\n",
    "print(\"say what?\"[2]) # the number in [] is the \"index\" position, starting from 0 (zero)\n",
    "print(\"say what?\"[2:6]) # the numbers in [] are the start index, \":\" UP TO, NOT INCLUDING, the second index\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# use what you know\n",
    "\n",
    "* create a Python script that starts with \"say what?\" and prints \"say something!\" using substring and \"+\" operators\n",
    "\n",
    "<p style=\"visibility:hidden\">\n",
    "    print(\"say what?\"[0:3] + \" something!\")\n",
    "</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"say what?\"[0:4] + \"something!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Transforming strings to digits and digits to strings\n",
    "\n",
    "NOTE: \"100\" and 100 are NOT the same in Python (unlike in Perl!).  \"100\" is a string, and 100 is an integer.\n",
    "\n",
    "So this does not work:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(1 + \" more time!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Therefore, you might need to force them to represent themselves as numbers or as strings.\n",
    "\n",
    "In Python, there are methods you can call on the various kinds of data-type to achieve this.  \n",
    "\n",
    "    int() = \"to integer\"\n",
    "    float() = \"to float\"\n",
    "    str() = \"to string\"\n",
    "\n",
    "For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(str(1) + \" more time\")  # from integer to string\n",
    "print(float(\"1\") + 3)  # from string to float\n",
    "print(float(1) + 3)  # from integer to float\n",
    "print(int(\"1\") + 3)  # from string to float\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prove that you understand\n",
    "\n",
    "Create some python that creates the following output:\n",
    "\n",
    "\n",
    "\n",
    "    Dear Mark Wilkinson,\n",
    "    \n",
    "         You are the\n",
    "              #1 teacher in the world!\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Variables and Constants\n",
    "\n",
    "Other than the reserved words, you can use any set of characters as your variables.  In Python, variables are usually written in lower\\_case, using a \\_ to separate words.  Other languages, like Java, use \"CamelCase\".  Please don't do this - it will annoy other Python programmers!  :-)\n",
    "\n",
    "The rules are:\n",
    "\n",
    "* Variables may start with a '_' or a letter character\n",
    "* after the starting character, it may be letters or digits or underscores \n",
    "* variables are case-sensitive (A_B is not the same as a_b)\n",
    "\n",
    "\n",
    "Constants (variables that SHOULD not change their value are indicated in CAPITAL_LETTERS... however, unlike some other programming languages, Python does not enforce this rule!  (keep out of my yard :-) )\n",
    "\n",
    "**SCOPE**\n",
    "\n",
    "Scope is an important concept.  A variable always exists in a particular \"scope\" - meaning, \"within a particular context within that program\".  Some variables will be \"global\" in their scope, others will be \"local\" to a particular function.\n",
    "\n",
    "We have not yet talked about 'functions', but just think of a 'function' as a sub-program inside of your program, that can be re-used many times by different parts of your software.\n",
    "\n",
    "Here is an example of scope.  You don't need to understand this code right now!  Just listen to what I tell you about it:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "a = 1  # assign the value of 1 to a\n",
    "b = 2  # assign 2 to b\n",
    "\n",
    "print (\"a is \" + str(a))\n",
    "print (\"b is \" + str(b))\n",
    "\n",
    "def add_two_numbers(x, y):\n",
    "    z = x + y\n",
    "    print(\"the sum is \" + str(z))\n",
    "    print(\"I can also see \" + str(a) + \" and \" + str(b) + \" inside of this function!\")\n",
    "\n",
    "add_two_numbers(a,b)\n",
    "\n",
    "print(\"z is \" + str(z))   # this fails!!  z doesn't exist!  (neither the variable, nor its value!)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Pre-defined variables and CONSTANTS\n",
    "\n",
    "We wont discuss all of these (there are many!), but will discuss only a few that are particularly important ones.  Many of these variables are important only in certain situations (for example, they can represent the context of a regular expression match, such as the content to the left and right of the match).    The critical ones for us right now are:   \n",
    "\n",
    "\n",
    "| Function/CONST|  meaning  |\n",
    "| -----  | ------  | \n",
    "|  sys.stderr  |   the filehandle for printing errors  |\n",
    "|  sys.stdin   |   the filehandle for input  |\n",
    "|  sys.stdout  |   the filehandle for standard output (usually the terminal window)  |\n",
    "|  False    |    the value \"false\"  |\n",
    "|  True   |    the value \"true\"  |\n",
    "|  None   |    the value \"does not exist\"  |\n",
    "|  sys.argv    |    the command-line arguments |\n",
    "  \n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "print (sys.argv)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ranges - a Python data-type similar to a constant\n",
    "\n",
    "A \"range\" is a sequence of numbers from \"start\" to \"stop\" with a particular \"step\" value between them.  Ranges in Python are slightly different from many other languages, because Python treats a range similar to a constant - it is \"one thing\", that has properties.\n",
    "\n",
    "ranges are specified using the <code>range()</code> command\n",
    "\n",
    "1..10    - from 1 to 10\n",
    "\n",
    "\"a\"..\"z\" - from a to z\n",
    "\n",
    "(1..10) === 5    - does the range of 1 to 10 include 5?  (true)\n",
    "\n",
    "(1..10) === 34    - does the range of 1 to 10 include 34? (false)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = range(1,10)  # numbers from 1 to 10, with INDEX POSITIONS starting from 0\n",
    "print(a)  # see, it is a \"thing\"\n",
    "print(list(a))  # to see its content, you pass it to functions or call methods\n",
    "print(a.index(7))  # where is the number \"7\" in this range?\n",
    "print(a[3])  # what is the third element of the range?\n",
    "# you can also create ranges from ranges\n",
    "print(a[1:4])\n",
    "print(\"\")\n",
    "print(\"range 0 to 30 step 3\")\n",
    "print(list(range(0,30,3)))\n",
    "\n",
    "#why use a range?\n",
    "\n",
    "a = range(1,2)\n",
    "b = range(1,100000000000000000000000)\n",
    "\n",
    "# that is much easier than a = [1,2,3,4,5,6,7,8,9,10,......1000000000000000000]\n",
    "# also smaller memory footprint\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>\n",
    "\n",
    "\n",
    "</pre>\n",
    "\n",
    "# Regular Expressions\n",
    "\n",
    "Regular expressions are a tool that allow you to do complex searches of text.  It is a set of characters that are used to encode a \"pattern\" that can then be applied to a piece of text looking for things that match the pattern.\n",
    "\n",
    "When I started my career in bioinformatics, I used regular expressions *every day*, so we will spend a lot of time on this section.  They are incredibly useful, especially in a field like biology where much of the information is in the form of large, complex \"wordy\" descriptive documents.\n",
    "\n",
    "For example, this is a GenBank record:\n",
    "     \n",
    "<code>\n",
    "LOCUS       SCU49845     5028 bp    DNA             PLN       21-JUN-1999\n",
    "DEFINITION  Saccharomyces cerevisiae TCP1-beta gene, partial cds, and Axl2p\n",
    "            (AXL2) and Rev7p (REV7) genes, complete cds.\n",
    "ACCESSION   U49845\n",
    "VERSION     U49845.1  GI:1293613\n",
    "KEYWORDS    .\n",
    "SOURCE      Saccharomyces cerevisiae (baker's yeast)\n",
    "  ORGANISM  Saccharomyces cerevisiae\n",
    "            Eukaryota; Fungi; Ascomycota; Saccharomycotina; Saccharomycetes;\n",
    "            Saccharomycetales; Saccharomycetaceae; Saccharomyces.\n",
    "REFERENCE   1  (bases 1 to 5028)\n",
    "  AUTHORS   Torpey,L.E., Gibbs,P.E., Nelson,J. and Lawrence,C.W.\n",
    "  TITLE     Cloning and sequence of REV7, a gene whose function is required for\n",
    "            DNA damage-induced mutagenesis in Saccharomyces cerevisiae\n",
    "  JOURNAL   Yeast 10 (11), 1503-1509 (1994)\n",
    "  PUBMED    7871890\n",
    "REFERENCE   2  (bases 1 to 5028)\n",
    "  AUTHORS   Roemer,T., Madden,K., Chang,J. and Snyder,M.\n",
    "  TITLE     Selection of axial growth sites in yeast requires Axl2p, a novel\n",
    "            plasma membrane glycoprotein\n",
    "  JOURNAL   Genes Dev. 10 (7), 777-793 (1996)\n",
    "  PUBMED    8846915\n",
    "REFERENCE   3  (bases 1 to 5028)\n",
    "  AUTHORS   Roemer,T.\n",
    "  TITLE     Direct Submission\n",
    "  JOURNAL   Submitted (22-FEB-1996) Terry Roemer, Biology, Yale University, New\n",
    "            Haven, CT, USA\n",
    "FEATURES             Location/Qualifiers\n",
    "     source          1..5028\n",
    "                     /organism=\"Saccharomyces cerevisiae\"\n",
    "                     /db_xref=\"taxon:4932\"\n",
    "                     /chromosome=\"IX\"\n",
    "                     /map=\"9\"\n",
    "     CDS             <1..206\n",
    "                     /codon_start=3\n",
    "                     /product=\"TCP1-beta\"\n",
    "                     /protein_id=\"AAA98665.1\"\n",
    "                     /db_xref=\"GI:1293614\"\n",
    "                     /translation=\"SSIYNGISTSGLDLNNGTIADMRQLGIVESYKLKRAVVSSASEA\n",
    "                     AEVLLRVDNIIRARPRTANRQHM\"\n",
    "     gene            687..3158\n",
    "                     /gene=\"AXL2\"\n",
    "     CDS             687..3158\n",
    "                     /gene=\"AXL2\"\n",
    "                     /note=\"plasma membrane glycoprotein\"\n",
    "                     /codon_start=1\n",
    "                     /function=\"required for axial budding pattern of S.\n",
    "                     cerevisiae\"\n",
    "                     /product=\"Axl2p\"\n",
    "                     /protein_id=\"AAA98666.1\"\n",
    "                     /db_xref=\"GI:1293615\"\n",
    "                     /translation=\"MTQLQISLLLTATISLLHLVVATPYEAYPIGKQYPPVARVNESF\n",
    "                     TFQISNDTYKSSVDKTAQITYNCFDLPSWLSFDSSSRTFSGEPSSDLLSDANTTLYFN\n",
    "         </code>\n",
    "         \n",
    "                     \n",
    "What is the gene name for this record?  \n",
    "\n",
    "If I give you 10 of these, could you give me the gene names for those 10?  \n",
    "\n",
    "If I gave you 35,000 (i.e. ~the number of genes in the human genome) could youi give me the gene names for all of them?  \n",
    "\n",
    "This is why we need regular expressions - to look for patterns in text in order to extract specific information from massive volumes of records.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Regular expression structure in Python\n",
    "\n",
    "There is [a good tutorial](https://www.tutorialspoint.com/python3/python_reg_expressions.htm) on regular expressions at TutorialsPoint.  It goes into more detail than I will here.\n",
    "\n",
    "Regular expressions are applied using the \"Regular Expression\" object (called \"re\").   \n",
    "\n",
    "A \"search\" function can be called, that takes a regular expression patten, and a piece of text as arguments, plus some additional \"flags\" that modify how the match is executed.\n",
    "\n",
    "<code> re.search(pattern, text, flags) </code>\n",
    "     \n",
    "FIRST, I will show you one somewhat complicated regular expression just to show you what they can do.  Then we will learn step-by-step how to build them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#!/usr/bin/python3\n",
    "import re  # this brings the python regular expression object into your program\n",
    "\n",
    "text = \"Cats are smarter than dogs\"\n",
    "\n",
    "matchObj = re.search( r'(ca\\w+) are .*?(d.*)', text, re.M|re.I)  # this should match \"Cats\" and \"dogs\n",
    "\n",
    "if matchObj:\n",
    "   print (\"matchObj.group(0) : \", matchObj.group(0))  \n",
    "   print (\"matchObj.group() : \", matchObj.group())  # group() can also be used instead of group(0)\n",
    "   print (\"matchObj.group(1) : \", matchObj.group(1))\n",
    "   print (\"matchObj.group(2) : \", matchObj.group(2))\n",
    "else:\n",
    "   print (\"No match!!\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Simple searches\n",
    "\n",
    "The most simple use of regular expressions are for \"search\" - does the pattern exist?  Yes or no.  We will now search for for simple characters.  The letter \"A\", the number \"5\", the character \">\".  The regular expression pattern for these is straightforward:\n",
    "\n",
    "<code>  r'A' </code>      <code>  r'5' </code>      <code>  r'>' </code>  \n",
    "\n",
    "Search happens from \"left\" to \"right\".\n",
    "\n",
    "For example:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#!/usr/bin/python3\n",
    "import re  # this brings the python regular expression object into your program\n",
    "\n",
    "text = \"We will learn <A LOT> in the next 2 weeks!  I hope...\"\n",
    "print(text)\n",
    "\n",
    "\n",
    "matchObj = re.search( r'A', text)  # this should match \"A\"\n",
    "\n",
    "if matchObj:\n",
    "   print (\"match r'A' : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "matchObj = re.search( r'2', text)  # this should match \"2\"\n",
    "\n",
    "if matchObj:\n",
    "   print (\"match r'2' : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "matchObj = re.search( r'>', text)  # this should match \">\"\n",
    "\n",
    "if matchObj:\n",
    "   print (\"match r'>' : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "matchObj = re.search( r'}', text)  # this should fail to match \"}\" because } doesn't exist in the string\n",
    "\n",
    "if matchObj:\n",
    "   print (\"match r'}' : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match  r'}'\")\n",
    "\n",
    "# can also search for multiple characters\n",
    "matchObj = re.search( r'A LOT', text)  # this should match \"A LOT\" \n",
    "if matchObj:\n",
    "   print (\"match r'A LOT' : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Now you try\n",
    "\n",
    "* Create the text \"Selection of axial growth sites in yeast requires Axl2p\"\n",
    "* create a regular expression to match the protein name\n",
    "* match \"yeast requires\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 'arbitrary' character - matching a little more complex\n",
    "\n",
    "So far, regular expressions don't seem very useful.  We have to already know the name of the protein before we can match it!  How do we match things that we *don't* already know?\n",
    "\n",
    "Regular expressions have special symbols that represent 'kinds' of characters.  For example:\n",
    "\n",
    "<code>\n",
    "        \\w --> a 'word' character (a-z A-Z 0-9 _)\n",
    "        \\W --> a 'non-word' character (everthing EXCEPT a word character)\n",
    "        \\d --> a 'digit' (0-9)\n",
    "        \\D --> a non-digit\n",
    "        \\s --> a space (newline, return, space, tab)\n",
    "        \\S --> non-space (anything other than a space)\n",
    "        .  --> ANY character EXCEPT the newline (\\n) character.\n",
    "        ^  --> anchor to the beginning (the regular expression MUST match the first character of the text)\n",
    "        $  --> anchor to the end (the regular expression MUST match the last character of the text)\n",
    "        \\b --> 'word boundary'  (the invisible space between the end of a word and the space character after the word)\n",
    "        \\t --> tab\n",
    "        \\n --> newline\n",
    "        \\r --> return\n",
    "        \\  --> make a special character non-special (e.g. \\. means \"period\")\n",
    "        [a-z] --> the range of lowercase letters from a to z\n",
    "        [A-Z] --> the range of uppercase letters from A to Z\n",
    "        [0-9] --> the range of digits from 0 to 9\n",
    "</code>\n",
    "\n",
    "As before, matches are read from left to right, and the first match found is reported.\n",
    "\n",
    "For example:\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#!/usr/bin/python3\n",
    "import re  # this brings the python regular expression object into your program\n",
    "\n",
    "text = \"We will learn <A LOT> in the next 2 weeks!  I hope... yes\"\n",
    "print(text)\n",
    "\n",
    "\n",
    "matchObj = re.search( r'\\w', text)  # this should match the first 'word character'\n",
    "if matchObj:\n",
    "   print (\"match r'\\w' : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n",
    "matchObj = re.search( r'\\w\\w', text)  # this should match the first two word characters\n",
    "if matchObj:\n",
    "   print (\"match  r'\\w\\w' : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n",
    "matchObj = re.search( r'\\d', text)  # this should match the first digit\n",
    "if matchObj:\n",
    "   print (\"match r'\\d' : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n",
    "matchObj = re.search( r'\\d\\d', text)  # this should match the first two digits (fail!)\n",
    "if matchObj:\n",
    "   print (\"match  r'\\d\\d' :  \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match of \\d\\d!!\")\n",
    "\n",
    "\n",
    "\n",
    "matchObj = re.search( r'\\d\\s\\w', text)  # this should match digit, space, word character\n",
    "if matchObj:\n",
    "   print (\"match r'\\d\\s\\w': \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n",
    "\n",
    "matchObj = re.search( r'\\S\\S\\s\\S\\S\\S\\S', text)  # this should match the first instance of 2 nonspace, space, 4 nonspace\n",
    "if matchObj:\n",
    "   print (\"match r'\\S\\S\\s\\S\\S\\S\\S' : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n",
    "matchObj = re.search( r'>............', text)  # this should match the first instance  '2' and a bunch of characters\n",
    "if matchObj:\n",
    "   print (\"match r'>............' : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n",
    "\n",
    "matchObj = re.search( r'[A-Z]', text)  # this should match the the first capital letter\n",
    "if matchObj:\n",
    "   print (\"match r'[A-Z]': \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n",
    "matchObj = re.search( r'[a-z]', text)  # this should match the first lowercase letter\n",
    "if matchObj:\n",
    "   print (\"match r'[a-z]' : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n",
    "\n",
    "matchObj = re.search( r'^[a-z]', text)  # this should match the first lowercase letter AT THE BEGINNING OF THE TEXT\n",
    "if matchObj:\n",
    "   print (\"match ^[a-z] : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match of lowercase at beginning of text!!\")\n",
    "\n",
    "\n",
    "\n",
    "matchObj = re.search( r'[a-z]$', text)  # this should match the lowercase letter AT THE END OF THE TEXT\n",
    "if matchObj:\n",
    "   print (\"match r'[a-z]$' : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n",
    "\n",
    "matchObj = re.search( r'\\.\\w', text)  # this should match a period followed by a word\n",
    "if matchObj:\n",
    "   print (\"match r'\\.\\w' : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match of period followed by word!!\")\n",
    "\n",
    "\n",
    "matchObj = re.search( r'\\..\\w', text)  # this should match a period followed by 2 X anything \n",
    "if matchObj:\n",
    "   print (\"match r'\\..\\w' : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match of period followed by word!!\")\n",
    "\n",
    "matchObj = re.search( r'\\w{5}', text)  # match a word character 5 times\n",
    "if matchObj:\n",
    "   print (\"match r'\\w{5}' : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match of five characters\")\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Try some more matches for yourself!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## test yourself\n",
    "\n",
    "      selection of axial growth sites in yeast requires Axl2p\n",
    "\n",
    "* find the gene name in this sentence\n",
    " * assume that gene names are always 5 characters long, \n",
    " * gene names are at the end of a sentence\n",
    " * gene name patters are three letters, a number, and a letter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "text=\"selection of axial growth sites in yeast requires Axl2p\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Matching more than once\n",
    "\n",
    "There are other symbols that have a special meaning.  \n",
    "<code>         \n",
    "`     + --> the preceding pattern should match \"one or more times\"`\n",
    "`     * --> the preceding pattern should match \"zero or more times\"`\n",
    "`     ? --> optional OR \"limit to the smallest match possible\"`\n",
    "</code>\n",
    "\n",
    "They are used in this way:\n",
    "\n",
    "<code>\n",
    "`    r'\\w+'  --> one or more word characters`\n",
    "`    r'\\d*'  --> zero or more digits`\n",
    "`    r'\\s*\\w'  --> zero or more spaces, followed by a word character`\n",
    "</code>\n",
    "\n",
    "These special symols have surprising (and often bad!) behaviors!  They are probably the hardest part to understand...\n",
    "\n",
    "See their behavior below:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "text = 'selection of axial       growth sites in yeast requires Axl2p'\n",
    "\n",
    "\n",
    "matchObj = re.search( r'\\w+', text)  # this should match the first word of any length \n",
    "if matchObj:\n",
    "   print (\"match r'\\w+' : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n",
    "matchObj = re.search( r'\\w+$', text)  # this should match the last word of any length\n",
    "if matchObj:\n",
    "   print (\"match r'\\w+$' : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n",
    "\n",
    "matchObj = re.search( r'axial\\s*\\w+', text)  # this should match 'axial', any number of spaces (including zero!), and the next word\n",
    "if matchObj:\n",
    "   print (\"match r'axial\\s*\\w+' : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n",
    "print()\n",
    "\n",
    "print(\"TRYING r'.*' --> match everything\")\n",
    "matchObj = re.search( r'.*', text)  # this should match any character any number of times (i.e. everything!)\n",
    "if matchObj:\n",
    "   print (\"match : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n",
    "print(\"TRYING r'.*\\s' --> maximal match\")\n",
    "matchObj = re.search( r'.*\\s', text)  # this should match any character any number of times, followed by a space\n",
    "if matchObj:\n",
    "   print (\"match : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n",
    "print(\"TRYING r'.*?\\s'  --> minimal match\")\n",
    "matchObj = re.search( r'.*?\\s', text)  # any character any number of times, followed by a space, smallest match possible\n",
    "if matchObj:\n",
    "   print (\"match : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## matching this or that, or anything other than this or that\n",
    "\n",
    "It is possible to be more selective than \"any character\".  Maybe you only want words that start with a vowel!  (why?  I have no idea... ;-) )\n",
    "\n",
    "To make these kinds of expressions, use square brackets.  For example:  <code> [aeiou] </code>\n",
    "\n",
    "To do the OPPOSITE, use square brackets with a '^'.  For example:  <code> [^aeiou] </code> (not aeio or u)\n",
    "\n",
    "You can also say \"or\" for multiple characters using a '|' (\"pipe\").  For example <code> cat|dog </code>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "text = 'selection of axial       growth sites in yeast requires Axl2p'\n",
    "\n",
    "matchObj = re.search( r'[aeiou]\\w+', text)  # aeiou followed by the rest of the word\n",
    "if matchObj:\n",
    "   print (\"match r'[aeiou]\\w+' : \", matchObj.group(), \"(not what we expected?)\")\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n",
    "matchObj = re.search( r'\\b[aeiou]\\w+', text)  # word boundary, aeiou, followed by the rest of the word\n",
    "if matchObj:\n",
    "   print (\"match r'\\\\b[aeiou]\\w+': \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n",
    "matchObj = re.search( r'\\b[^aeiou]\\w+', text)  # word boundary, not a vowel, followed by the rest of the word\n",
    "if matchObj:\n",
    "   print (\"match r'\\\\b[^aeiou]\\w+' : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n",
    "matchObj = re.search( r'\\b[AEIOU]\\w+', text)  # word boundary, capital vowel, followed by the rest of the word\n",
    "if matchObj:\n",
    "   print (\"match r'\\\\b[AEIOU]\\w+' : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n",
    "matchObj = re.search( r'axial|growth', text)  # axial or growth (matches the first success)\n",
    "if matchObj:\n",
    "   print (\"match r'axial|growth' : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n",
    "matchObj = re.search( r'nothing|growth', text)    # nothing or growth (matches the first success)\n",
    "if matchObj:\n",
    "   print (\"match r'nothing|growth' : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Capturing sub-portions of what you match\n",
    "\n",
    "You can use round brackets () to capture parts of regular expressions.  These become elements of the .groups function.\n",
    "\n",
    "For example:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "text = \"A: we will learn regular expressions. B: we will apply that knowledge.\"\n",
    "\n",
    "\n",
    "matchObj = re.search( r'[A-Z]:\\s(.*?\\.)', text)  # match the index letter, then CAPTURE the rest of the sentence\n",
    "if matchObj:\n",
    "   print (\"match : \", matchObj.group())\n",
    "   print (\"match group 1 : \", matchObj.group(1))\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n",
    "matchObj = re.search( r'[A-Z]:\\s(.*?\\.).*?[A-Z]:\\s(.*?\\.)', text)  # notice the .*?  ....remember for later\n",
    "if matchObj:\n",
    "   print (\"match : \", matchObj.group())\n",
    "   print (\"match group 1 : \", matchObj.group(1))\n",
    "   print (\"match group 2 : \", matchObj.group(2))\n",
    "else:\n",
    "   print (\"No match!!\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Case-insensitive matches and multi-line matches\n",
    "\n",
    "We haven't discussed the \"flags\" that are the third element that can be passedto a regular expression search.\n",
    "\n",
    "There are a variety of flags (you can look them up yourself!), but we will only discuss two of them:  Case insenitive search, and multi-line search.  These are represented by the 'flags' re.I (case-insensitive) and re.S (multi-line)\n",
    "\n",
    "Case insensitive example:\n",
    "\n",
    "<code>\n",
    "    matchObj = re.search( r'birne', text, re.I) # matches Birne or birne or bIRne or...\n",
    "</code>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "text = 'selection of axial       growth sites in yeast requires Axl2p'\n",
    "\n",
    "matchObj = re.search( r'axl2p', text, re.I)  # axl2p regardless of capital letters\n",
    "if matchObj:\n",
    "   print (\"match : \", matchObj.group())\n",
    "else:\n",
    "   print (\"No match!!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>\n",
    "\n",
    "</pre>\n",
    "Multi-line matches require a bit more explanation.  Remember the definition of the <code> . </code> symbol?  It was \"matches anything except newline\".  We should first see what that means:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "\n",
    "text = \"\"\"A: we will learn regular expressions. \n",
    "B: we will apply that knowledge.\"\"\"\n",
    "\n",
    "# this is exactly the same regular expression as in the previous example, but now the text is split into two lines\n",
    "matchObj = re.search( r'[A-Z]:\\s(.*?\\.).*?[A-Z]:\\s(.*?\\.)', text)  \n",
    "if matchObj:\n",
    "   print (\"match group 1 : \", matchObj.group(1))\n",
    "   print (\"match group 2 : \", matchObj.group(2))\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>\n",
    "\n",
    "</pre>\n",
    "Why is there no match?   Because the <code>.</code> symbol does not match the newline character --> the regular expression doesn't cross the newline --> the second match \"[B]: we wil apply that knowledge\" never happens.\n",
    "\n",
    "the <code>re:S</code> changes the definition of <code>.</code> so that it now also matches newline (i.e. it matches EVERYTHING!)\n",
    "\n",
    "Let's try again with the multiline (S) flag:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "\n",
    "text = \"\"\"A: we will learn regular expressions. \n",
    "B: we will apply that knowledge.\"\"\"\n",
    "\n",
    "# this is exactly the same regular expression as in the previous example, but now the text is split into two lines\n",
    "matchObj = re.search( r'[A-Z]:\\s(.*?\\.).*?[A-Z]:\\s(.*?\\.)', text, re.S)  \n",
    "if matchObj:\n",
    "   print (\"match group 1 : \", matchObj.group(1))\n",
    "   print (\"match group 2 : \", matchObj.group(2))\n",
    "else:\n",
    "   print (\"No match!!\")\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>\n",
    "\n",
    "\n",
    "</pre>\n",
    "## Now you try\n",
    "\n",
    "* use Regular Expressions to extract the Genbank Reference Sequence identifier (NM_115294.6) from the first FASTA record below\n",
    "* use Regular Expressions to extract the gene name (AP3) from the title line of the first record\n",
    "* use Regular Expressions to extarct the nucleotide sequence from the first FASTA record\n",
    "* **prove that your Regular Expressions are \"good\" by extracting the SAME INFORMATION from the second FASTA record **  (Do you remember the very first day we talked about \"abstraction\"?  This is another example of abstraction - a single regular expression that matches every possible case)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>\n",
    ">NM_115294.6 Arabidopsis thaliana K-box region and MADS-box transcription factor family protein (AP3), mRNA\n",
    "AAAAAAATCAGTTTACATAAATGGAAAATTTATCACTTAGTTTTCATCAACTTCTGAACTTACCTTTCAT\n",
    "GGATTAGGCAATACTTTCCATTTTTAGTAACTCAAGTGGACCCTTTACTTCTTCAACTCCATCTCTCTCT\n",
    "TTCTATTTCACTTCTTTCTTCTCATTATATCTCTTGTCCTCTCCACCAAATCTCTTCAACAAAAAGATTA\n",
    "AACAAAGAGAGAAGAATATGGCGAGAGGGAAGATCCAGATCAAGAGGATAGAGAACCAGACAAACAGACA\n",
    "AGTGACGTATTCAAAGAGAAGAAATGGTTTATTCAAGAAAGCACATGAGCTCACGGTTTTGTGTGATGCT\n",
    "AGGGTTTCGATTATCATGTTCTCTAGCTCCAACAAGCTTCATGAGTATATCAGCCCTAACACCACAACGA\n",
    "AGGAGATCGTAGATCTGTACCAAACTATTTCTGATGTCGATGTTTGGGCCACTCAATATGAGCGAATGCA\n",
    "AGAAACCAAGAGGAAACTGTTGGAGACAAATAGAAATCTCCGGACTCAGATCAAGCAGAGGCTAGGTGAG\n",
    "TGTTTGGACGAGCTTGACATTCAGGAGCTGCGTCGTCTTGAGGATGAAATGGAAAACACTTTCAAACTCG\n",
    "TTCGCGAGCGCAAGTTCAAATCTCTTGGGAATCAGATCGAGACCACCAAGAAAAAGAACAAAAGTCAACA\n",
    "AGACATACAAAAGAATCTCATACATGAGCTGGAACTAAGAGCTGAAGATCCTCACTATGGACTAGTAGAC\n",
    "AATGGAGGAGATTACGACTCAGTTCTTGGATACCAAATCGAAGGGTCACGTGCTTACGCTCTTCGTTTCC\n",
    "ACCAGAACCATCACCACTATTACCCCAACCATGGCCTTCATGCACCCTCTGCCTCTGACATCATTACCTT\n",
    "CCATCTTCTTGAATAATTAAAGGCTAAAAGGTTTGCTGGTGCCATCATTGTCTATCTAATTATTTAGTAA\n",
    "CTACTTAAAACATAAGGCATGGTGTTGCTAAAACCTTAAACTGTCATGTTTCTTAGTTATGTATTTTAAA\n",
    "GCCTAAAGAAATATGGATTGTGTGATCAGTAGTGCTTAGGCTTATTGTGTGTGGAATGTTTTCAAGACTT\n",
    "TTATCATGTATCGTATTATTATATTGACCACTCTACTTAATTATGCTACAAATTTACTCGATTTGATTTT\n",
    "CTACTTGAATGCATATATATTGTC\n",
    "\n",
    "\n",
    "\n",
    ">NM_127349.4 Arabidopsis thaliana Homeodomain-like superfamily protein (WUS), mRNA\n",
    "CTCTCACACAAAACCTAAAATCTCTTTACTACCAGCAAGTTGTTTTCTTGCTAACTTCAAACTTCTCTTT\n",
    "CTCTTGTTCCTCTCTAAGTCTTGATCTTATTTACCGTTAACTTTGTGAACAAAAGTCGAATCAAACACAC\n",
    "ATGGAGCCGCCACAGCATCAGCATCATCATCATCAAGCCGACCAAGAAAGCGGCAACAACAACAACAACA\n",
    "AGTCCGGCTCTGGTGGTTACACGTGTCGCCAGACCAGCACGAGGTGGACACCGACGACGGAGCAAATCAA\n",
    "AATCCTCAAAGAACTTTACTACAACAATGCAATCCGGTCACCAACAGCCGATCAGATCCAGAAGATCACT\n",
    "GCAAGGCTGAGACAGTTCGGAAAGATTGAGGGCAAGAACGTCTTTTACTGGTTCCAGAACCATAAGGCTC\n",
    "GTGAGCGTCAGAAGAAGAGATTCAACGGAACAAACATGACCACACCATCTTCATCACCCAACTCGGTTAT\n",
    "GATGGCGGCTAACGATCATTATCATCCTCTACTTCACCATCATCACGGTGTTCCCATGCAGAGACCTGCT\n",
    "AATTCCGTCAACGTTAAACTTAACCAAGACCATCATCTCTATCATCATAACAAGCCATATCCCAGCTTCA\n",
    "ATAACGGGAATTTAAATCATGCAAGCTCAGGTACTGAATGTGGTGTTGTTAATGCTTCTAATGGCTACAT\n",
    "GAGTAGCCATGTCTATGGATCTATGGAACAAGACTGTTCTATGAATTACAACAACGTAGGTGGAGGATGG\n",
    "GCAAACATGGATCATCATTACTCATCTGCACCTTACAACTTCTTCGATAGAGCAAAGCCTCTGTTTGGTC\n",
    "TAGAAGGTCATCAAGAAGAAGAAGAATGTGGTGGCGATGCTTATCTGGAACATCGACGTACGCTTCCTCT\n",
    "CTTCCCTATGCACGGTGAAGATCACATCAACGGTGGTAGTGGTGCCATCTGGAAGTATGGCCAATCGGAA\n",
    "GTTCGCCCTTGCGCTTCTCTTGAGCTACGTCTGAACTAGCTCTTACGCCGGTGTCGCTCGGGATTAAAGC\n",
    "TCTTTCCTCTCTCTCTCTCTTTCGTACTCGTATGTTCACAACTATGCTTCGCTAGTGATTAATGATGCAG\n",
    "TTGTTATATTAGTAGTTAACTAGTTATCTCTCGTTATGTGTAATTTGTAATTACTAGCTAAGTATCGTCT\n",
    "AGGTTTTAATTGTAATTGACAACCGTTTTATCTCTATGATGAATAAGTTAAAATTTTA\n",
    "\n",
    "</pre>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Structures\n",
    "\n",
    "So far we have talked about simple data types (strings, numbers, ranges, and regular expressions).  Now we will talk about higher level data structures - lists, \"dictionaries\", etc.\n",
    "\n",
    "\n",
    "## Lists\n",
    "\n",
    "A list in python is a series of values separated by commas, and enclosed in square brackets.  For example\n",
    "\n",
    "    a = [1,2,3,4,5]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = [1,2,3,4,5]\n",
    "print(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "It is *usually* the case that you will only put the same KIND of value into a list.  e.g. a list of numbers or a list of strings.  This is not *always* true, but please believe me when I tell you - if you don't follow this rule, your code will not be as reliable as it should be!\n",
    "\n",
    "But, just to prove that it is OK:\n",
    "\n",
    "    a = [1, 2, \"three\", 4, 5]\n",
    "    \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = [1,2,\"three\",4,5]\n",
    "print(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Every element of a list has an \"address\" or an \"index position\".  In Python, index position numbers start with '0' so the first element of the list is element index '0', the third element is element index '2', etc.  To see the individual elements, you put the index number after the variable:\n",
    "\n",
    "    a = [\"one\", \"two\", \"three\", \"four\", \"five\"]\n",
    "    print(a[2])\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = [\"one\", \"two\", \"three\", \"four\", \"five\"]\n",
    "print(a[2])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### List Slices\n",
    "\n",
    "A \"slice\" is a sub-list inside of a list.  The notation for a slice is:  <code>index_start:index_end:step</code>\n",
    "\n",
    "for example:\n",
    "\n",
    "    a = [\"one\", \"two\", \"three\", \"four\", \"five\"]\n",
    "    print(a[2:4])\n",
    "    print(a[4:2:-1])\n",
    "\n",
    "**The start index is INCLUDED in the output, the end index IS NOT INCLUDED in the output.**  Be careful!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = [\"one\", \"two\", \"three\", \"four\", \"five\"]\n",
    "print(a[2:4])  # this is index 2 and 3!!  NOT 2, 3 and 4!!!\n",
    "print(a[4:2:-1])  # note how easy it is to make mistakes with list indexes!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "It is possible to specify only the start, or only the end index, by eliminating the other.  For example:\n",
    "\n",
    "    a = [\"one\", \"two\", \"three\", \"four\", \"five\"]\n",
    "    print(a[2:])  # print from index 2 until the end\n",
    "    print(a[:4])  # print from index 0 until index 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = [\"one\", \"two\", \"three\", \"four\", \"five\"]\n",
    "print(a[2:])  # print from index 2 until the end\n",
    "print(a[:4])  # print from index 0 until index 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### Modify list values\n",
    "\n",
    "You can modify a list value, or a slice value, in the same way as assigning any other variable.  Simply assign a value to that index position.  For example:\n",
    "\n",
    "    a = [\"one\", \"two\", \"three\", \"four\", \"five\"]\n",
    "    print(a[2])\n",
    "    a[2] = \"THREE\"  # switch it to capital letters\n",
    "    print(a[2])\n",
    "    \n",
    "    a = [\"one\", \"two\", \"three\", \"four\", \"five\"]\n",
    "    print(a[2:3])\n",
    "    a[2] = [\"TWO\", \"THREE\"]  # NOTE that you are assiging a list, because a slice is a list!\n",
    "    print(a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = [\"one\", \"two\", \"three\", \"four\", \"five\"]\n",
    "print(a[2])\n",
    "a[2] = \"THREE\"  # switch it to capital letters\n",
    "print(a[2])\n",
    "\n",
    "print()\n",
    "\n",
    "a = [\"one\", \"two\", \"three\", \"four\", \"five\"]\n",
    "print(a[2:4])\n",
    "a[2:4] = \"THREE\", \"FOUR\"\n",
    "print(a)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "<p style=\"color:red;\">Again I will tell you:  BE VERY CAREFUL WITH SLICES!!  Especially if you are going to modify the values of a slice!  Watch how easily we can get confused an completely ruin our data!</p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = [\"one\", \"two\", \"three\", \"four\", \"five\"]\n",
    "print(a[2:3])    # THIS IS NOT A SLICE WITH TWO ELEMENTS!!!\n",
    "a[2:3] = \"THREE\", \"FOUR\"   # HERE WE ADD TWO ELEMENTS...  oops!!!\n",
    "print(a)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(  a[9]  )# doesn't exist!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a[9] = \"nine\"  # this also fails! ... you cannot update an index that doesn't exist\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = a + [\"six\", \"seven\", \"eight\", \"nine\"]\n",
    "print(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## other list operations\n",
    "\n",
    "You can add to a list using the <code>.append()</code> method.\n",
    "\n",
    "You can remove a specific value from a list using teh <code>.remove()</code> method.\n",
    "\n",
    "You can reverse a list using the <code>.reverse()</code> method.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mylist = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]\n",
    "\n",
    "print(mylist)\n",
    "mylist.append(11)\n",
    "print(mylist)\n",
    "\n",
    "mylist.remove(2)\n",
    "print(mylist)\n",
    "\n",
    "mylist.reverse()\n",
    "print(mylist)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## YOU TRY\n",
    "\n",
    "* create the list <code>[\"one\", \"two\", \"three\", \"four\", \"five\"]</code>\n",
    "* change the value of \"five\" into the number \"5\"\n",
    "* change it back again (to \"five\")\n",
    "* invert the elements \"one\" and \"two\" so that the list is now <code>[\"two\", \"one\", \"three\", \"four\", \"five\"]</code>\n",
    "* invert the elements \"three\" and \"four\" **WITHOUT TYPING THE WORDS \"three\" or \"four\"** in your code!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mylist = [\"one\", \"two\", \"three\", \"four\", \"five\"]\n",
    "# invert the elements \"three\" and \"four\" \n",
    "#WITHOUT TYPING THE WORDS \"three\" or \"four\" in your code!\n",
    "\n",
    "mylist[2:4]= mylist[3:1:-1]\n",
    "print(mylist)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "listolist = []\n",
    "listolist.append([\"my\", \"dog\", \"has\", \"fleas\"])\n",
    "listolist.append([\"my\", \"cat\", \"has\", \"piojos?\"])\n",
    "print(listolist)\n",
    "print(listolist[0][1])\n",
    "print(listolist[1][1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# \"Dictionary\" data structures\n",
    "\n",
    "(Note:  In many languages (not Python) this data structure is called a \"hash\", not a \"dictionary.  They are almost the same thing...)\n",
    "\n",
    "A Dictionary is a set of key/value pairs.  For example, imagine we needed to create a dictionary of the ages of people in the class:\n",
    "\n",
    "     Mark =  50\n",
    "     Jonas = 25\n",
    "     Alberto = 24\n",
    "     SmartGuy = 16\n",
    "     \n",
    "This is what a Dictionary does.  The syntax for the dictionary data structure is:\n",
    "\n",
    "    a = {key1: value1 ,  key2: value2  ,  key3: value3 .......}\n",
    "    \n",
    "Keys and Values can be any basic data type, data structure, or variable  eg. for a key string, it is \"key1\"\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dict = {'key1': 1 ,  'key2': 2  ,  'key3': 3}\n",
    "print(dict)\n",
    "\n",
    "x = \"keyX\"\n",
    "y = \"keyY\"\n",
    "z = \"keyZ\"\n",
    "\n",
    "dict = {x: 1 ,  y: 2  ,  z: 3}  # variables are expanded to whatever it is...\n",
    "print(dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# Indexes of a Dictionary\n",
    "\n",
    "This is exactly like an Array, but you use the key instead of an index number.  For example:\n",
    "\n",
    "    x = \"keyX\"\n",
    "    y = \"keyY\"\n",
    "    z = \"keyZ\"\n",
    "    a = {x: 1 ,  y: 2  ,  z: 3}\n",
    "    \n",
    "    print(  a[\"keyX\"]   )\n",
    "    \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = \"keyX\"\n",
    "y = \"keyY\"\n",
    "z = \"keyZ\"\n",
    "dict = {x: 1 ,  y: 2  ,  z: 3}\n",
    "\n",
    "print( \"keyX has the value: \", dict[\"keyX\"]   )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Patient name:  Mark\n",
    "#Patient age:  27\n",
    "#Patient weight: 190kg\n",
    "\n",
    "thispatient = {'name': \"Mark\",  'age': 27, 'weight': '190kg'}\n",
    "\n",
    "print(\"This patient's name is: \", thispatient['name'])\n",
    "print(\"This patient's weight is: \", thispatient['weight'])\n",
    "print(\"This patient's age is: \", thispatient['age'])\n",
    "\n",
    "\n",
    "children = ['jay', 'silent bob']\n",
    "\n",
    "patient1 = {'name': \"Mark\",  'age': 27, 'weight': '190kg'}\n",
    "patient2 = {'name': \"Silvia\",  'age': 29, 'weight': '120kg'}\n",
    "patient3 = {'name': \"Julio\",  'age': 35, 'weight': '290kg', 'mychildren': children}\n",
    "\n",
    "patients = {1: patient1,  2:patient2, 3:patient3}\n",
    "print(patients)\n",
    "print(patients[2]['name'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# assigning values\n",
    "\n",
    "Just as with a List, you can assign a new value to a Dictionary:\n",
    "\n",
    "    dict[\"keyX\"] = \"XXX\"\n",
    "    print(dict)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "dict[\"keyX\"] = \"XXXXXXXXX\"\n",
    "print(dict)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " ### but note that we have more flexibility with a Dictionary!  Watch..\n",
    "    \n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dict[\"Key Doesn't Exist\"] = \"ha!  This works!\"\n",
    "print(dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### deleting values\n",
    "\n",
    "If you want to delete a Dictionary entry, use the \"del\" function:\n",
    "    \n",
    "    del dict[\"Key Doesn't Exist\"]\n",
    "    print(dict)\n",
    "    \n",
    "if you want to eleminate all key/values from the Dictionary (but maintain the dictionary variable!) use the \"clear\" \"method\" (we will discuss \"methods\" when we talk about object oriented programming):\n",
    "\n",
    "    dict.clear()\n",
    "    print(dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "del dict[\"Key Doesn't Exist\"]\n",
    "print(dict)\n",
    "\n",
    "dict.clear()\n",
    "print(dict)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## more complex data structures\n",
    "\n",
    "imagine we have two classes of students, and we want to create a dictionary from those two classes...  how?\n",
    "\n",
    "    class1 = {\"Mark\": 50,  \"Jonas\": 25,  \"Alberto\": 24, \"SmartGuy\": 16}\n",
    "    class2 = {\"Julio\": 55,  \"Laura\": 23,  \"Angela\": 25, \"SmartWoman\": 14}\n",
    "\n",
    "    classes = {\"Morning\": class1,  \"Afternooon\": class2}\n",
    "\n",
    "    print(classes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class1 = {\"Mark\": 50,  \"Jonas\": 25,  \"Alberto\": 24, \"SmartGuy\": 16}\n",
    "class2 = {\"Julio\": 55,  \"Laura\": 23,  \"Angela\": 25, \"SmartWoman\": 14}\n",
    "\n",
    "classes = {\"Morning\": class1,  \"Afternooon\": class2}\n",
    "\n",
    "print(classes)\n",
    "print()\n",
    "\n",
    "print(\"Mark, from the morning class, has age:  \",  classes[\"Morning\"][\"Mark\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# NOW YOU\n",
    "\n",
    "Imagine that we want to be able to say \"hello\" or \"goodbye\" in either English, Spanish, or German.  Create a SINGLE dictionary that ***abstracts this problem***\n",
    "\n",
    "    English ==>  \"hello\", \"bye!\"\n",
    "    Spanish ==>  \"hola\", \"hasta luego!\"\n",
    "    German ==>   \"hallo\", \"tschuess!\"\n",
    "\n",
    "Your code should include two variables:  \n",
    "* language (the value of this variable is either \"English\", \"Spanish\", or \"German\")\n",
    "* event (the value of this variable is either \"greeting\", or \"departure\")\n",
    "\n",
    "I want to access the corect phrase for \"English greeting\", or \"German departure\"\n",
    "\n",
    "**This is a challenge!!!  If you get it right, then you deeply understand both dictionaries, and abstraction!  :-)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# put your code here...\n",
    "\n",
    "p1= {'greeting': \"hello\", 'departure': \"bye!\"}\n",
    "p2= {'greeting': \"hola\", 'departure': \"hasta luego!\"}\n",
    "p3= {'greeting': \"hallo\", 'departure':\"tschuess!\"}\n",
    "phrases = {'English': p1, 'Spanish': p2, 'German': p3}\n",
    "#print(phrases)\n",
    "\n",
    "language = 'Spanish'\n",
    "event = 'departure'\n",
    "\n",
    "print(phrases[language][event])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Operators\n",
    "\n",
    "| operator | definition | example |\n",
    "| --- | --- | --- |\n",
    "| ASSIGNMENT |\n",
    "| = | assign value b to a | a = b |\n",
    "| += | add and assign | a+=b --> a = a + b |\n",
    "| -= | subtract and assign | a-=b --> a = a - b |\n",
    "| \\*= | multiply and assign | a\\*=b --> a = a * b |\n",
    "| \\= | divide and assign |  a\\=b --> a = a \\ b |\n",
    "| %= | modulus and assign | a%=b --> a = **remainder** of a\\b |\n",
    "| \\*\\*= | exponent and assign | a\\*\\*=b  --> a = a\\*\\*b |\n",
    "| MATHEMATICS (returns output of operation) |\n",
    "| + | plus | a + b |\n",
    "| - | minus | a - b |\n",
    "| * | multiply | a * b |\n",
    "| \\ | divide | a\\b |\n",
    "| \\*\\* | exponent | a\\*\\*b |\n",
    "| % | modulus | a % b --> the **remainder** after a\\b | \n",
    "| COMPARISON (returns TRUE or FALSE [except <=>]) |\n",
    "| == | equals? | a == b |\n",
    "| != | not equals? | a != b |\n",
    "| > | greater than? | a > b |\n",
    "| < | less than? | a < b | \n",
    "| >= | greater than or equal? | a >= b |\n",
    "| <= | less than or equal? | a <= b |\n",
    "| <=> | combined comparison | a <=> b: returns 0 if a = b, 1 if a > b, and -1 if a < b |\n",
    "| LOGIC  (say a = 1 b = 2), and **in order of operator precedence**|\n",
    "| & | Logical AND: if both are non-zero | a & b --> TRUE |\n",
    "| &#124; | Logical OR: if either are non-zero | a &#124; b --> TRUE |\n",
    "| ! | Logical NOT:  invert a true to false, false to true | !(a &#124;&#124; b) --> FALSE | "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}