Image Support in Completion Apps

[www-xu]

I meet a scenario that I need to understand a image in completion apps. But after I read documents, I found only chat apps allow use upload images. Do you have any plans to support this in completion apps? Thanks!

[mmabrouk]

Hi @www-xu ,

We want to support images in completion. The issue is not technical but more design / UX of the feature.

The interface for the chat service is a dictionary of input strings in addition to a list of messages in the OpenAI format. We added images as part of the messages as is done in OpenAI.
However, in completion, the interface is only a dictionary of input strings. We do not send messages. That is why we could not add images.

Can you please share your use case, and why do you need to use images in the completion playground (why can't you use these in the chat playground).

What kind of prompt are you creating? What is the fixed part (the prompt template) and what is the stuff in the test set?
Is it so, that the image is part of the prompt template (the fixed part)? Or are you looking to have a PromptTemplate with a User message that includes an image_url which is a variable that is specified in the inputs?

[www-xu]

Hi @www-xu ,

We want to support images in completion. The issue is not technical but more design / UX of the feature.

The interface for the chat service is a dictionary of input strings in addition to a list of messages in the OpenAI format. We added images as part of the messages as is done in OpenAI. However, in completion, the interface is only a dictionary of input strings. We do not send messages. That is why we could not add images.

Can you please share your use case, and why do you need to use images in the completion playground (why can't you use these in the chat playground).

What kind of prompt are you creating? What is the fixed part (the prompt template) and what is the stuff in the test set? Is it so, that the image is part of the prompt template (the fixed part)? Or are you looking to have a PromptTemplate with a User message that includes an image_url which is a variable that is specified in the inputs?

Thanks for your reply! In my scenario, I wanna get key points of a image. I think it's more about the last use case you listed, I wann have a PromptTemplate with a User message that includes an image_url which is a variable that is specified in the inputs. And I will try to use chat playground. Thanks again!