gradio chat exception

[faev999]

hi all,
I had the following exception when trying to run the gradio example with: python -m mlx_vlm.chat_ui --model mlx-community/Qwen2-VL-72B-Instruct-4bit

.../mlx_vlm/chat_ui.py", line 32, in chat
    if len(message.files) >= 1:
    
AttributeError: 'dict' object has no attribute 'files'

when using the CLI example with: python -m mlx_vlm.generate --model mlx-community/Qwen2-VL-72B-Instruct-4bit --max-tokens 100 --temp 0.0 --image http://images.cocodataset.org/val2017/000000039769.jpg there's no exception.

I installed the package with: pip install mlx-vlm and have tried python 3.12 and python 3.10 with and got the same result

[kaimerklein]

I modified the chat function in chat_ui.py like this:

def chat(message, history, temperature, max_tokens):
    chat = []
    if len(message["files"]) >= 1:
        chat.append(message["text"])
    else:
        raise gr.Error("Please upload an image. Text only chat is not supported.")

    files = message["files"][-1]
    if model.config.model_type != "paligemma":
        messages = apply_chat_template(processor, config, message["text"], num_images=1)
    else:
        messages = message["text"]

    response = ""
    for chunk in stream_generate(
        model, processor, files, messages, image_processor, max_tokens, temp=temperature
    ):
        response += chunk
        yield response

Seems to work.

[faev999]

I also modified that function like that, but then I got other kind of errors:

...prompt_utils.py", line 138, in apply_chat_template
if p.get("role") in ["system", "assistant"]:
AttributeError: 'str' object has no attribute get

Did you also modify that file?

I modified the chat function in chat_ui.py like this:

def chat(message, history, temperature, max_tokens):
    chat = []
    if len(message["files"]) >= 1:
        chat.append(message["text"])
    else:
        raise gr.Error("Please upload an image. Text only chat is not supported.")

    files = message["files"][-1]
    if model.config.model_type != "paligemma":
        messages = apply_chat_template(processor, config, message["text"], num_images=1)
    else:
        messages = message["text"]

    response = ""
    for chunk in stream_generate(
        model, processor, files, messages, image_processor, max_tokens, temp=temperature
    ):
        response += chunk
        yield response

Seems to work.

[faev999]

I also modified that function like that, but then I got other kind of errors:

...prompt_utils.py", line 138, in apply_chat_template
if p.get("role") in ["system", "assistant"]:
AttributeError: 'str' object has no attribute get

Did you also modify that file?

I modified the chat function in chat_ui.py like this:

def chat(message, history, temperature, max_tokens):
    chat = []
    if len(message["files"]) >= 1:
        chat.append(message["text"])
    else:
        raise gr.Error("Please upload an image. Text only chat is not supported.")

    files = message["files"][-1]
    if model.config.model_type != "paligemma":
        messages = apply_chat_template(processor, config, message["text"], num_images=1)
    else:
        messages = message["text"]

    response = ""
    for chunk in stream_generate(
        model, processor, files, messages, image_processor, max_tokens, temp=temperature
    ):
        response += chunk
        yield response

Seems to work.

I added a string type check for p like this and now it works:

else:
           for prompts in prompt:
                for i, p in enumerate(prompts):
                    if not isinstance(p, str):
                        messages.append(process_single_prompt(p, i == 0))
                        if p.get("role") in ["system", "assistant"]:
                            messages.append(process_single_prompt(p, False))
                    else:
                        is_first = i == 0 or i == 1
                        messages.append(process_single_prompt(p, is_first))