Use source NER pipeline component with spacy-legacy (config file)

[m-nlp-q]

I am trying to continue the training of el_core_news_lg, so in the training config file in [components] section I use the source specifier for both ner and tok2vec components. When I run debug config command I get the error:

catalogue.RegistryError: Cant't find 'spacy.Tok2Vec.v1' in registry spacy -> architectures. Available names: spacy-legacy.MaxoutWindowEncoder.v1, spacy-legacy.MishWindowEncoder.v1, spacy-legacy.TextCatEnsemble.v1, spacy-legacy.Tok2Vec.v1, spacy.CharacterEmbed.v1, spacy.EntityLinker.v1, spacy.HashEmbedCNN.v1, spacy.MaxoutWindowEncoder.v2, spacy.MishWindowEncoder.v2, spacy.MultiHashEmbed.v1, spacy.PretrainCharacters.v1, spacy.PretrainVectors.v1, spacy.Tagger.v1, spacy.TextCatBOW.v1, spacy.TextCatCNN.v1, spacy.TextCatEnsemble.v2, spacy.TextCatLowData.v1, spacy.Tok2Vec.v2, spacy.Tok2VecListener.v1, spacy.TorchBiLSTMEncoder.v1, spacy.TransitionBasedParser.v1, spacy.TransitionBasedParser.v2

That is because the trained component use a deprecated version of Tok2Vec component architecture, and specifically it uses spacy.Tok2Vec.v1 . The spacy-legacy package solves this incompatibility problems, I checked that it is installed (ver 3.0.1), but I still get the error.

I also tried to replace

[components.ner]
source = "el_core_news_lg"

with

[components.ner]
source = "el_core_news_lg"

[components.ner.model]
@architectures = "spacy.TransitionBasedParser.v2"
state_type = "ner"
extra_state_tokens = false
hidden_width = 64
maxout_pieces = 2
use_upper = true
nO = null

[components.ner.model.tok2vec]
@architectures = "spacy-legacy.Tok2Vec.v1"

which is the section that describes the ner component in the config file of el_core_news_lg, replacing spacy.Tok2Vec.v1 with spacy-legacy.Tok2Vec.v1.I did the same for the tok2vec component but still get the same error.

Do I have to override from spacy.Tok2Vec.v1 to spacy-legacy.Tok2Vec.v1 manually, or should spacy-legacy package do it automatically, and if so, what am I doing wrong?

This is my config file:

[paths]
train = "/content/model_inputs/training_configuration/train.spacy"
dev = "/content/model_inputs/training_configuration/dev.spacy"
vectors = null
init_tok2vec = null
vocab_data = null

[system]
gpu_allocator = null
seed = 0

[nlp]
lang = "el"
pipeline = ["tok2vec","ner"]
batch_size = 1000
disabled = []
before_creation = null
after_creation = null
after_pipeline_creation = null
tokenizer = {"@tokenizers":"spacy.Tokenizer.v1"}

[components]

[components.ner]
source = "el_core_news_lg"

[components.ner.model]
@architectures = "spacy.TransitionBasedParser.v2"
state_type = "ner"
extra_state_tokens = false
hidden_width = 64
maxout_pieces = 2
use_upper = true
nO = null

[components.ner.model.tok2vec]
@architectures = "spacy-legacy.Tok2Vec.v1"

[components.tok2vec]
source = "el_core_news_lg"

[components.tok2vec.model]
@architectures = "spacy-legacy.Tok2Vec.v1"

[corpora]

[corpora.dev]
@readers = "spacy.Corpus.v1"
limit = 0
max_length = 0
path = ${paths:dev}
gold_preproc = false
augmenter = null

[corpora.train]
@readers = "spacy.Corpus.v1"
path = ${paths:train}
max_length = 5000
gold_preproc = false
limit = 0
augmenter = null

[training]
train_corpus = "corpora.train"
dev_corpus = "corpora.dev"
seed = ${system:seed}
gpu_allocator = ${system:gpu_allocator}
dropout = 0.1
accumulate_gradient = 1
patience = 5000
max_epochs = 0
max_steps = 0
eval_frequency = 1000
frozen_components = []
before_to_disk = null

[training.batcher]
@batchers = "spacy.batch_by_words.v1"
discard_oversize = false
tolerance = 0.2
get_length = null

[training.batcher.size]
@schedules = "compounding.v1"
start = 100
stop = 1000
compound = 1.001
t = 0.0

[training.logger]
@loggers = "spacy.WandbLogger.v1"
project_name = "spacy-v3.0.0a2"
remove_config_values = []

[training.optimizer]
@optimizers = "Adam.v1"
beta1 = 0.9
beta2 = 0.999
L2_is_weight_decay = true
L2 = 0.01
grad_clip = 1.0
use_averages = true
eps = 0.00000001
learn_rate = 0.001

[training.score_weights]
sents_p = null
sents_r = null
ents_per_type = null
sents_f = 0.0
ents_f = 1.0
ents_p = 0.0
ents_r = 0.0

[initialize]
vocab_data = ${paths.vocab_data}
vectors = ${paths.vectors}
init_tok2vec = ${paths.init_tok2vec}
before_init = null
after_init = null

[initialize.components]

[initialize.components.ner]

[initialize.components.ner.labels]
@readers = "spacy.read_labels.v1"
path = "/content/corpus/ner.json"
require = false

[initialize.lookups]
@misc = "spacy.LookupsDataLoader.v1"
lang = ${nlp.lang}
tables = ["lexeme_norm"]

[initialize.tokenizer]

[svlandeg]

Hi! Apologies for the late follow-up.

Do I have to override from spacy.Tok2Vec.v1 to spacy-legacy.Tok2Vec.v1 manually, or should spacy-legacy package do it automatically, and if so, what am I doing wrong?

This should in fact happen automatically. I tried reproducing your issue by copy-pasting the config as you had it, but I can't reproduce the error message you're getting. What specific versions of spacy and spacy-legacy do you have installed?

When looking into your config file, there's one other point I'd like to note. The components that you're sourcing, will be taken as is from the models. That also means that you don't have to specify the model architecture of these components - this architecture can't be changed at this point, because the components are already trained.

So, in short, you can just remove the entire [components.ner.model] and [components.tok2vec.model] from your config file.

This shouldn't really interfer with the problem about spacy-legacy though, so happy to continue discussing that in more detail. One thing you could try if you're still running into this, is upgrading both spacy and spacy-legacy to the latest versions.

[h-amg]

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/__main__.py", line 4, in <module>
    setup_cli()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/cli/_util.py", line 69, in setup_cli
    command(prog_name=COMMAND)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/typer/main.py", line 497, in wrapper
    return callback(**use_params)  # type: ignore
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/cli/package.py", line 47, in package_cli
    package(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/cli/package.py", line 103, in package
    meta = get_meta(input_dir, meta)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/cli/package.py", line 276, in get_meta
    reqs = get_third_party_dependencies(nlp.config, exclude=existing_reqs)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/cli/package.py", line 213, in get_third_party_dependencies
    func_info = sub_registry.find(func_name)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/catalogue/__init__.py", line 148, in find
    func = self.get(name)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/catalogue/__init__.py", line 95, in get
    raise RegistryError(
catalogue.RegistryError: Cant't find 'spacy.MaxoutWindowEncoder.v1' in registry spacy -> architectures. Available names: spacy-legacy.CharacterEmbed.v1, spacy-legacy.HashEmbedCNN.v1, spacy-legacy.MaxoutWindowEncoder.v1, spacy-legacy.MishWindowEncoder.v1, spacy-legacy.MultiHashEmbed.v1, spacy-legacy.TextCatBOW.v1, spacy-legacy.TextCatCNN.v1, spacy-legacy.TextCatEnsemble.v1, spacy-legacy.Tok2Vec.v1, spacy-legacy.TransitionBasedParser.v1, spacy.CharacterEmbed.v2, spacy.EntityLinker.v1, spacy.HashEmbedCNN.v2, spacy.MaxoutWindowEncoder.v2, spacy.MishWindowEncoder.v2, spacy.MultiHashEmbed.v2, spacy.PretrainCharacters.v1, spacy.PretrainVectors.v1, spacy.SpanCategorizer.v1, spacy.Tagger.v1, spacy.TextCatBOW.v2, spacy.TextCatCNN.v2, spacy.TextCatEnsemble.v2, spacy.TextCatLowData.v1, spacy.Tok2Vec.v2, spacy.Tok2VecListener.v1, spacy.TorchBiLSTMEncoder.v1, spacy.TransitionBasedParser.v2

I was able to resolve the above issue by downgrading spacy from 3.1.3 to 3.1.0

[svlandeg]

Thanks for reporting back. I do think that in a clean install, this should work as well with the latest versions of both spacy and spacy-legacy, but I'm happy that you've found a way that works for you.