Skip to content

prob = -20.0 for all lemmas in en_core_web_lg #6119

Discussion options

You must be logged in to vote

Because they're no longer used as model features in v2, the probability tables aren't distributed with the models as of v2.3 to save space. The table is provided through spacy-lookups-data instead. See the section "Probability and cluster features" here for how to load them into an existing model: https://spacy.io/usage/v2-3#migrating

Note that the probability and cluster tables in spacy-lookups-data only include 1M tokens instead of 1.3M as in v2.2 models, so if you need to replicate the exact probability tables from v2.2, you'll want to export all the probabilities from a v2.2 model and import them in a custom v2.3 model.

Replies: 2 comments

Comment options

You must be logged in to vote
0 replies
Answer selected by ines
Comment options

You must be logged in to vote
0 replies
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
v2 spaCy v2.x
2 participants
Converted from issue

This discussion was converted from issue #6119 on December 10, 2020 23:57.