04_04_introduction_to_custom_pipes.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <center>Creating Location Pipe for Holocaust NER</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center>Dr. W.J.B. Mattingly</center>\n",
    "\n",
    "<center>Smithsonian Data Science Lab and United States Holocaust Memorial Museum</center>\n",
    "\n",
    "<center>January 2021</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Key Concepts in this Notebook"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1) How to add pipes to a spaCy model<br>\n",
    "2) How to custom labels to NER pipe<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, we will cover how to create a blank spaCy model, load in pipes from another model, save that new model. We will also work with custom entities."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating a Blank spaCy Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Whenever you are creating a custom spaCy model, it is usually a good idea to start with a blank model. Let's do that first. The spaCy blank class takes one argument, the language. In our case, we want to create a blank English model. The language affects how spaCy will tokenize the text."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import spacy\n",
    "\n",
    "main_nlp = spacy.blank(\"en\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our goal now is to take the NER pipeline from the spaCy small English model and inject it into our blank model First. We need to grab the English model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "en_model = spacy.load(\"en_core_web_sm\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we need to grab specifically the NER pipe. We can do that with the function get_pipe(). This will take one argument, the pipe desired. Because we want to grab the NER pipe, we will specifcy \"ner\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "ner = en_model.get_pipe(\"ner\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have that pipe, we can add it to our blank spaCy model using the add_pipe() function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "main_nlp.add_pipe(ner)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have that added to our main_nlp pipeline, let's test it out and make sure that it works."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Bob PERSON\n",
      "the United States GPE\n"
     ]
    }
   ],
   "source": [
    "sent = \"Hello, my name is Bob and I live in the United States.\"\n",
    "\n",
    "doc = main_nlp(sent)\n",
    "for ent in doc.ents:\n",
    "    print (ent.text, ent.label_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that our blank model now can use the ner pipe from the small English spaCy model. Success."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Saving a spaCy Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Saving a spaCy model is just as simple. Let's go ahead and save our blank model before moving forward. We can do this with the to_disk() function. This will take one argument, the location we want to save it to. We are going to put it in the subfolder models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "main_nlp.to_disk(\"models/sample_model\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Placing Pipes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes, we will want to make more sophistocated pipelines where we have multiple pipes. In these scenarios, we will want to place specific pipes in specific locations. Let's try and work through how to do that.\n",
    "\n",
    "We can do this several different ways. First, we can place pipes in order in our script. The last pipe added will be the last pipe in the pipeline. In the code below, we add English and German NERs to our pipeline. If we do this, however, we will get an error:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "[E007] 'ner' already exists in pipeline. Existing names: ['ner']",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-12-51f12b1879b3>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m      7\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      8\u001b[0m \u001b[0mmain_nlp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0madd_pipe\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0men_ner\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 9\u001b[1;33m \u001b[0mmain_nlp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0madd_pipe\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mde_ner\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;32mc:\\python38\\lib\\site-packages\\spacy\\language.py\u001b[0m in \u001b[0;36madd_pipe\u001b[1;34m(self, component, name, before, after, first, last)\u001b[0m\n\u001b[0;32m    339\u001b[0m             \u001b[0mname\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mutil\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mget_component_name\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mcomponent\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    340\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mname\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mpipe_names\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 341\u001b[1;33m             \u001b[1;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mErrors\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mE007\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mname\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mname\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mopts\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mpipe_names\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    342\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0msum\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mbool\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mbefore\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mbool\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mafter\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mbool\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mfirst\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mbool\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlast\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;33m>=\u001b[0m \u001b[1;36m2\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    343\u001b[0m             \u001b[1;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mErrors\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mE006\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mValueError\u001b[0m: [E007] 'ner' already exists in pipeline. Existing names: ['ner']"
     ]
    }
   ],
   "source": [
    "import spacy\n",
    "\n",
    "main_nlp = spacy.blank(\"en\")\n",
    "\n",
    "en_ner = spacy.load(\"en_core_web_sm\").get_pipe(\"ner\")\n",
    "de_ner = spacy.load(\"de_core_news_sm\").get_pipe(\"ner\")\n",
    "\n",
    "main_nlp.add_pipe(en_ner)\n",
    "main_nlp.add_pipe(de_ner)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The reason we are getting this error is because we are trying to add two NER pipes that have the same name, \"ner\". We need to give each of these a unique name. We can do this with the code below and then we will iterate over the pipeline to make sure we have all the pipes listed correctly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('en_ner', <spacy.pipeline.pipes.EntityRecognizer object at 0x000001A20BD97700>)\n",
      "('de_ner', <spacy.pipeline.pipes.EntityRecognizer object at 0x000001A20FADCA00>)\n"
     ]
    }
   ],
   "source": [
    "import spacy\n",
    "\n",
    "main_nlp = spacy.blank(\"en\")\n",
    "\n",
    "en_ner = spacy.load(\"en_core_web_sm\").get_pipe(\"ner\")\n",
    "de_ner = spacy.load(\"de_core_news_sm\").get_pipe(\"ner\")\n",
    "\n",
    "main_nlp.add_pipe(en_ner, name=\"en_ner\")\n",
    "main_nlp.add_pipe(de_ner, name=\"de_ner\")\n",
    "\n",
    "for pipe in main_nlp.pipeline:\n",
    "    print (pipe)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice in the output that we not only get the correct pipes with unique names, we also get them in the order we expected. Let's try and put the German model in front of the English one. To do that, we can pass an argument. You can use either before or after, depending on how you want to place the pipe. We will use before=\"en_ner\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('de_ner', <spacy.pipeline.pipes.EntityRecognizer object at 0x000001A212600FA0>)\n",
      "('en_ner', <spacy.pipeline.pipes.EntityRecognizer object at 0x000001A207E00580>)\n"
     ]
    }
   ],
   "source": [
    "import spacy\n",
    "\n",
    "main_nlp = spacy.blank(\"en\")\n",
    "\n",
    "en_ner = spacy.load(\"en_core_web_sm\").get_pipe(\"ner\")\n",
    "de_ner = spacy.load(\"de_core_news_sm\").get_pipe(\"ner\")\n",
    "\n",
    "main_nlp.add_pipe(en_ner, name=\"en_ner\")\n",
    "main_nlp.add_pipe(de_ner, name=\"de_ner\", before=\"en_ner\")\n",
    "\n",
    "for pipe in main_nlp.pipeline:\n",
    "    print (pipe)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Placement of specific pipes in a specific order is essential because later pipes receive the data of the earlier pipes. In the case of NER, this means that later pipes cannot overwrite earlier pipes, unless we give them permission."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Adding Custom Labels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to best demonstrate how to add custom labels, it is best to watch the video below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try to create some custom pipelines for fun. See how they perform on texts when ordered in specific ways."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Video"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div align=\"center\">\n",
       "<iframe width=\"560\" height=\"315\" src=\"https://www.youtube.com/embed/1l3v2Zcgb3s\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen></iframe>\n",
       "</div>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%html\n",
    "<div align=\"center\">\n",
    "<iframe width=\"560\" height=\"315\" src=\"https://www.youtube.com/embed/1l3v2Zcgb3s\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen></iframe>\n",
    "</div>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}