Using Spacy V2 en_core_web_lg-2.3.1 model in Spacy V3

[udaypk]

Hello folks,

We have been using Spacy V2 (en_core_web_lg-2.3.1) for POS tagging in our application. When we recently upgraded to Spacy V3, we found that the tagger of V3 is not as accuracte as V2. For our application, en_core_web_lg-2.3.1 tagger is 4% more accurate than en_core_web_lg-3.0.0

Can anyone suggest if there is a way to use en_core_web_lg-2.3.1 model in Spacy 3.

Thanks
Uday

PS: Spacy V3 tagger is producing many errors for auxillary verbs and gerunds.

[adrianeboyd]

That's interesting to hear about the tagger performance. Is this for fine-grained (token.tag) or coarse-grained (token.pos) tags?

In our internal evaluation the tagger accuracy for fine-grained tags was very similar overall (v2.3.1: 97.2; v3.0.0: 97.4), so I'm not immediately sure what might be causing the difference. If capitalization were a factor (proper vs. common nouns, for instance), I know of a few differences in the training setup that might account for it, but for verbs I'm not sure.

Unfortunately, there's no way to use the spaCy v2 model with spaCy v3. There are too many internal changes, primarily the ML library thinc used by spaCy was entirely rewritten for spaCy v3.

[udaypk]

Thanks for the quick response!

We were computing the difference in accuracy for coarse-grained (token.pos). Will check fine-grained. Can you point me to where fine-grained to coarse-grained conversion is happening in the code.

Thanks

[adrianeboyd]

Ah, if it's token.pos, there may some differences there for auxiliary verbs. We tried to get the mapping closer to the UD guidelines by incorporating information from the dependency parse if it's available, so copulas in particular may be different, as AUX in v3 instead of as VERB in v2. This distinction isn't present in the token.tag PTB tag set, but you can use the dependency parse to identify copulas. If you don't have a parser in your pipeline, the mapping should be pretty similar to v2, so you can try disabling the parser to test that out.

In any case, check the token.tag tags and if they are correct, then it's a matter of adjusting the rules that convert tag -> pos to the patterns you want. It should be fairly easy to configure, or to return to something closer to the v2.x settings if you want.

The mapping happens in the attribute_ruler and you can see all the patterns with:

ar = nlp.get_pipe("attribute_ruler")
ar.patterns # list of patterns dicts

The attribute ruler uses matcher patterns to identify the tokens to modify: https://spacy.io/api/attributeruler

You can also discard the current rules and import the v2 tag map by importing TAG_MAP from spacy/lang/en/tag_map.py from spacy v2: https://github.com/explosion/spaCy/blob/v2.x/spacy/lang/en/tag_map.py. Then use attribute_ruler.load_from_tag_map(TAG_MAP) to convert the tag map to the attribute ruler patterns. There are the v2 morph rules in morph_rules.py, too, if that's relevant.

Additionally, there are some rules for other reasons that you may want to preserve (tags for whitespace tokens, some custom lemmas), depending on whether it affects your task.

[udaypk]

Thanks a lot. This was very useful. Disabling the parser and adding Spacy V2 TAG_MAP attributes to attribute_ruler got the Spacy V3 pos_ attributes very close to V2.