Configure vocab for CJK #906
Conversation
eu9ene
requested review from
hneiva
and removed request for
a team and
hneiva
This reverts commit a6c35bf.
|
I'm thinking now that the 64K vocab as a workaround of split 32K vocabs may end up slowing down the decoding process because the softmax will be more expensive. Also I remember in the past the s2s models struggling to learn with large vocabulary outputs like 80k. So I wonder if we will see something of this in the Transformer. We'll see what happens in this preliminary experiments, but I'm starting to be more inclined to split vocabs. |
yeah, I guess I'll rerun the experiment with the latest fixes and we can test it. I'll also work on #913 so that we can experiment with the split vocabs |
# Conflicts: # pipeline/alignments/align.py # pipeline/data/cjk.py # pipeline/data/dataset_importer.py # pipeline/data/download-mono.py # pipeline/data/requirements/data.in # poetry.lock # taskcluster/kinds/finetune-student/kind.yml # taskcluster/kinds/train-student/kind.yml # taskcluster/kinds/train-teacher/kind.yml # tests/test_alignments.py # tests/test_cjk.py # tests/test_data_importer.py # tests/test_training.py # utils/config_generator.py
closes #745