Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deepseek-r1-distill-qwen-7b (and seemingly other reasoning models) exclude initial <think> tag #4761

Open
RealJohnGalt opened this issue Feb 4, 2025 · 2 comments
Labels
bug Something isn't working unconfirmed

Comments

@RealJohnGalt
Copy link

RealJohnGalt commented Feb 4, 2025

LocalAI version:
464686a

Environment, CPU architecture, OS, and Version:
linux + vulkan

Describe the bug
I am excluding the proper think tags since I cannot get them to show due to markdown.
deepseek-r1-distill-qwen-7b (and other deepseek models I've tested) exclude the initial think tag, but include the "/think" tag after. This is creates an issue with some frontends such as open webui, since they recently added support for think tags and hiding/showing/giving stats on thinking.

To Reproduce
Ask any question, response begins with reasoning directly, and ends with /think but has no beginning think tag.

Expected behavior
I expect think to precede the thinking.

An example output from: why is the sky blue?

Okay, the user is asking, "why is the sky blue?" I remember that there are a few explanations for this, so I need to figure out which one is the most accurate based on the provided context.

Looking at the context, there are two sources with the same question. Both sources explain that the sky appears blue because of the scattering of sunlight by atmospheric molecules. When sunlight enters the Earth's atmosphere, it's made up of all the colors of the spectrum. As it travels through the air, shorter wavelengths like blue and violet are scattered more than the longer wavelengths like red and orange. This scattering makes the sky appear blue, especially when the sun is high in the sky.

The context also mentions that in polluted skies, small particles like dust can scatter more blue light, making the sky look more blue. However, the user's question is general, so I don't need to include that unless it's necessary. Also, I should cite the sources that provide this information. Since both sources are from Britannica, I'll use [britannica.com] for the context.

I should structure my answer to first explain the phenomenon, then mention the factors that contribute to the sky's color. I'll include the citation at the end to give credit to the source. I'll keep it concise and clear, making sure it addresses the user's question directly.

The sky appears blue primarily due to the scattering of sunlight by molecules in the Earth's atmosphere. Sunlight is composed of all the colors of the visible spectrum, but shorter wavelengths like blue and violet are scattered more than longer wavelengths like red and orange. This scattering effect is more pronounced when the sun is high in the sky, as the sunlight travels through a larger portion of the atmosphere. The result is that the sky appears blue to our eyes [britannica.com].

@RealJohnGalt RealJohnGalt added bug Something isn't working unconfirmed labels Feb 4, 2025
@RealJohnGalt
Copy link
Author

Removed with markdown, but the end of the second paragraph had the /think tag without any initial think tag used

@aotsukiqx
Copy link

aotsukiqx commented Feb 5, 2025

same issue, on my mac studio, distill qwen 32b, only got /think tag.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working unconfirmed
Projects
None yet
Development

No branches or pull requests

2 participants