Skip to content

Custom spaCy NER model not making expected predictions #12739

Discussion options

You must be logged in to vote

The issue here is- if the model recognizes a given code as the correct inference in one document, why is it not able to identify another similar looking code number as the correct inference, in another similar document.

Unfortunately there's no satisfying answer to this. The model relies contextual representations that incorporate information from up to four words of context on either side of the target token. The entity recognizer then goes through the words of the document as a state machine, and makes decisions about how to construct the entities based on the prior state and the contextual tokens.

When I'm trying to debug the entity recogniser I basically step through the decisions i…

Replies: 1 comment 2 replies

Comment options

You must be logged in to vote
2 replies
@MisbahKhan789
Comment options

@honnibal
Comment options

Answer selected by honnibal
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
training Training and updating models feat / ner Feature: Named Entity Recognizer
2 participants