Skip to content

French tokenization - iconsistent application of exceptions in FR_BASE_EXCEPTIONS & other unexpected tokenization #8920

e-nesse started this conversation in Language Support
Discussion options

You must be logged in to vote

Replies: 1 comment 6 replies

Comment options

You must be logged in to vote
6 replies
@e-nesse
Comment options

@adrianeboyd
Comment options

@e-nesse
Comment options

@e-nesse
Comment options

@lsmith77
Comment options

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lang / fr French language data and models feat / tokenizer Feature: Tokenizer
3 participants
Converted from issue

This discussion was converted from issue #8916 on August 10, 2021 11:27.