regexTutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Importing and Using the `re` Module"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To start, we need to import the `re` module"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `re.search()` function allows us to see if a string contains a particular pattern in which we're interested. The general syntax is `re.search(<pattern>,<string>)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search(r\"ttt\",\"aggtttcctttagttt\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If the pattern is found in the string, `re.search()` will return an `re.Match` object. If it's not found, `re.search()` will return the value `None`. This behavior allows us to use `re.search()` to execute logical tests."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if re.search(r\"ttt\",\"aggtttcctttagttt\"):\n",
    "    print(\"Pattern found!\")\n",
    "else:\n",
    "    print(\"Pattern not found!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `re.sub()` function allows us to do find-and-replace operations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "re.sub(r\"ttt\",\"TTT\",\"aggtttcctttagttt\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Often, we want to specify patterns that are more flexible. Here's an example where we are looking for a g, then any 3 nucleotides, then a c."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "match = re.search(r\"g...c\",\"aggtttcctttagttt\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take a look at what part of our DNA sequence matched our pattern."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "match.group()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, try this example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "match = re.search(r\"g...c\",\"attcgaagcaggtttcct\")\n",
    "match.group()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The pattern `gtttc` is still present in this sequence. Why did this search return `gaagc`?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "allMatches = re.findall(r\"g...c\",\"attcgaagcaggtttcct\")\n",
    "print(allMatches)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Special Characters and Wildcards"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here are some examples using other special and wildcard characters. Before you run each example, predict what the output will be."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# \\w matches 'word' characters\n",
    "match = re.search(r\"\\w\\w\",\"t s m9\")\n",
    "match.group()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# \\d matches digits\n",
    "match = re.search(r\"\\d\",\"t s m9\")\n",
    "match.group()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# \\s matches any whitespace character\n",
    "match = re.search(r\"\\s\\w\\d\",\"t s m9\")\n",
    "match.group()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ^ matches the beginning of the string\n",
    "match = re.search(r\"^\\w\\s\\w\",\"t s m9\")\n",
    "match.group()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# $ matches the end of the string\n",
    "match = re.search(r\"\\w$\",\"t s m9\")\n",
    "match.group()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# \\S matches any non-whitespace character\n",
    "match = re.search(r\"\\s\\S\\s\",\"t s m9\")\n",
    "match.group()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# \\ can be used to escape special characters\n",
    "match = re.search(r\"\\$\\d\",\"I have $1.\")\n",
    "match.group()\n",
    "# What happens if you remove the \\ before $?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can create custom wildcards by listing the potentially matching characters inside square brackets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Searching for an even digit followed by an odd digit\n",
    "match = re.search(r\"[2,4,6,8][1,3,5,7,9]\",\"1249787\")\n",
    "match.group()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Repetition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# + matches one or more instances of a character\n",
    "match = re.search(r\"A+\",\"GCTTTGGAAAGG\")\n",
    "match.group()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# {} can specify a particular number of repetitions\n",
    "match = re.search(r\"A{2}\",\"GCTTTGGAAAGG\")\n",
    "match.group()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Look carefully at the output from the cell above. Why does the result go all the way to the last G, instead of stopping at the first G? This happens because repetition characters are greedy. They'll find the biggest possible match."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# * matches zero or more instances of a character\n",
    "matchOne = re.search(r\"CT*G\",\"GCTTTGGAAAGG\")\n",
    "matchOne.group()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "matchTwo = re.search(r\"GT*G\",\"GCTTTGGAAAGG\")\n",
    "matchTwo.group()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ? matches 0 or 1 instances of a character\n",
    "matchOne = re.search(r\"GC?T\",\"GCTTTGGAAAGG\")\n",
    "matchOne.group()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "matchTwo = re.search(r\"TC?T\",\"GCTTTGGAAAGG\")\n",
    "matchTwo.group()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Repetition characters can also be used in combination with wildcards."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Matches any pattern with one or more characters between\n",
    "# a C and a G\n",
    "match = re.search(r\"C.+G\",\"GCTTTGGAAAGG\")\n",
    "match.group()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Capturing Text from Searches"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To capture specific characters embedded in our larger search pattern, we can wrap them in parentheses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Capturing two nucleotides just before or after stretches\n",
    "# of 2 or more As.\n",
    "match = re.search(r\"(\\w\\w)AA+(\\w\\w)\",\"GCTTTGCAAAAAGG\")\n",
    "match.group()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Looking at the captured text\n",
    "match.groups()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Capturing text can also be used with re.findall()\n",
    "matches = re.findall(r\"(\\w\\w)AA+(\\w\\w)\",\"GCTTTGCAAAAAGGTCTTAGAAAAATG\")\n",
    "print(matches)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Looking at just the captured text from the 2nd match\n",
    "print(matches[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Substitution with Captured Text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use text captured during a search as part of a subsitution. Here', we're adding vertical lines at the edges of our stretches of As. `\\1` indicates the first section of captured text, `\\2` the second section, etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "re.sub(r\"(\\w\\w)(AA+)(\\w\\w)\",r\"\\1|\\2|\\3\",\"GCTTTGCAAAAAGGTCTTAGAAAAATG\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}