You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[CMake] Enable SentencePiece tokenizer by default (#47)
The SentencePiece tokenizer was disabled by default to
reduce the binary size of the built library, while it causes
some error when users expect to use the SentencePiece
tokenizer.
This PR enables it by default. And we will need to manually
disable it if we need to reduce its binary size.
0 commit comments