Multi-word Entity

[YasineNifa]

Hello, I am new to Spacy, I would like to know how the model can recognize entities with 2 words.
For example in this sentence ("Mr. Best flew to New York on Saturday morning from New delhi to buy a New clothes."), the spacy recognizes "New York" and "New Delhi" as GPE entities.

I want to know how Spacy works ? Does it go through the sentence word by word and then compare the words with the words on the training dataset?
if it is the case ? how can detect the entity with 2 words like "New York" and "New Delhi"

Thank you

[adrianeboyd]

Most statistical NER models tag each word individually with tags that show whether a word is at the beginning/middle/end of an entity or not an entity (O). One common kind of sequence tags are IOB tags. The quality of this article is not great, but it has some useful examples and links: https://en.wikipedia.org/wiki/Inside%E2%80%93outside%E2%80%93beginning_(tagging)

What spacy stores internally in a doc is technically the IOB2 format, although it's often just called IOB:

import spacy
nlp = spacy.load("en_core_web_sm")

doc = nlp("Mary Smith flew to New York with Sarah.")
for t in doc:
    print(t.text, t.ent_iob_, t.ent_type_)

Output:

Mary B PERSON
Smith I PERSON
flew O 
to O 
New B GPE
York I GPE
with O 
Sarah B PERSON
. O 

I hope that helps!

[YasineNifa]

Yeah that's was so helpful thank you :)
I have an other question please.
I tried to annotate this sentence "I study at New York University"
In the first try, I write "University" with a capital letter and the spacy recognize the "New York University" as ORG
sentence1 = 'I study at New York University' => New York University ORG
In the second try. I write "university" with a lower "u" and spacy recognize "New York" as GPE
sentence2 = 'I am going to New York' => New York GPE.

I want to understand how spacy do recognize these entities ?