This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

Tokenizer for Blenderbot 2.0 #4066

Answered by klshuster

rvong65 asked this question in Q&A

rvong65
Oct 8, 2021

What tokenizer does blenderbot 2.0 use?

Answered by klshuster

BlenderBot 2.0 3B uses the bytelevelbpe tokenizer; BlenderBot 2.0 400m uses the gpt2 tokenizer

View full answer

Replies: 1 comment

klshuster
Oct 8, 2021

BlenderBot 2.0 3B uses the bytelevelbpe tokenizer; BlenderBot 2.0 400m uses the gpt2 tokenizer

0 replies

Answer selected by rvong65

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment