Upgrade tiktoken-rs

[hendrikvanantwerpen]

Version 0.6.0 of the tiktoken-rs crate was released. The new version makes some methods private that we used to get the raw bytes for tokens. This PR changes how we build OpenAI tokenizers from using tiktoken-rs to interning and reading the tiktoken data files directly.

Main changes:

• Move all tests to a separate crate, so that tests can depend on bpe-openai. This is similar to what we did for benchmarks before. Without this, we'd have to duplicate some of the logic to instantiate at least one OpenAI tokenizer in the bpe crate to be able to run tests.
• Add gzipped copies of the .tiktoken files to the bpe-openai crate and build serialized tokenizers from those.
• Rename bpe-openai public methods to the official token set names (e.g. cl100k_base instead of just cl100k).
• bpe got some functions to read tiktoken data, and does not depend (conditionally) on tiktoken-rs anymore.
• Upgrade all tiktoken-rs dependencies to version 0.6.

[aneubeck]

LGTM
Maybe add a comment somewhere why the tests live in another folder.
It's obviously not perfect that we now have to compile two crates first, before we can run any tests... But, I don't really see an easy way around that problem.