Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

get_tokenizer().apply_chat_template occurs an error #1663

Open
CodeABao opened this issue Feb 1, 2025 · 5 comments
Open

get_tokenizer().apply_chat_template occurs an error #1663

CodeABao opened this issue Feb 1, 2025 · 5 comments
Assignees
Labels
category: tokenizers Tokenizer class or submodule update
Milestone

Comments

@CodeABao
Copy link

CodeABao commented Feb 1, 2025

when i use model DeepSeek-R1-Distill-Qwen-7B bellow error occurs

RuntimeError: Exception from C:\Jenkins\workspace\private-ci\ie\build-windows-vs2019\b\repos\openvino.genai\src\cpp\src\tokenizer.cpp:546:
Chat template for the current model is not supported by Jinja2Cpp. Please apply template manually to your prompt before calling generate. For example: <start_of_turn>user{user_prompt}<end_of_turn><start_of_turn>model

tokenizer_config.json

@andrei-kochin
Copy link
Collaborator

Hello @CodeABao,
Thank you for reporting the issue!
That is known Jinja2Cpplimitation and we are going to workaround it soon inside the GenAI. So far please send the template manually and use the notebook as a ref.

@CodeABao
Copy link
Author

CodeABao commented Feb 5, 2025

Hello @CodeABao, Thank you for reporting the issue! That is known Jinja2Cpplimitation and we are going to workaround it soon inside the GenAI. So far please send the template manually and use the notebook as a ref.

thanks,

when i input "hi", the output is below.

Image

when i stop it and execute it again, the issues occurs.
Image

@Wovchena
Copy link
Collaborator

Wovchena commented Feb 5, 2025

I don't think stopping a cell and expecting that everything rolls back to a valid state is a good idea in general. Plus GenAI doesn't restore its state for sure if it was cancelled in any other way but callback.

@CodeABao
Copy link
Author

CodeABao commented Feb 5, 2025

I don't think stopping a cell and expecting that everything rolls back to a valid state is a good idea in general. Plus GenAI doesn't restore its state for sure if it was cancelled in any other way but callback.

okk, thanks.

@Wovchena
Copy link
Collaborator

Wovchena commented Feb 6, 2025

I stumbled at a similar issue with

Argument shapes are inconsistent.

So that particular fail doesn't relate to cell cancellation, it's a GenAI bug.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: tokenizers Tokenizer class or submodule update
Projects
None yet
Development

No branches or pull requests

4 participants