How to use gpt-4-vision-preview ?

[sbetzin]

Hi There,

love your work and this repo!
Tried to use gpt-4-vision-preview but only got an exception:
var result = await client.GetChatCompletions(message, model: "gpt-4-vision-preview");

And I got:

Thank you
Sebastian

[sbetzin]

Btw: Trying to test if I can get the vision to work via api:

var json = JsonSerializer.Serialize(new { type = "image_url", image_url = new { url = base64} });

var message = Dialog.StartAsSystem("Return image description.").ThenUser(json);

[rodion-m]

Hey @sbetzin ! Sorry for the late response. Vision feature isn't supported yet.

[sbetzin]

Hi, yes it is :) It was able to implement it with my own code

private static object CreateÎmageMessages(string systemPrompt, string prompt, double temperature, int maxTokens,
    string base64Image)
{
    // via https://platform.openai.com/docs/guides/vision
    var payload = new
    {
        model = "gpt-4-vision-preview",
        messages = new object[]
        {
                new
                {
                    role = "system",
                    content = new object[]
                    {
                        new
                        {
                            type = "text",
                            text = systemPrompt
                        }
                    }
                },
                new
                {
                    role = "user",
                    content = new object[]
                    {
                        new
                        {
                            type = "text",
                            text = prompt
                        },
                        new
                        {
                            type = "image_url",
                            image_url = new
                            {
                                url = "data:image/jpeg;base64," + base64Image,
                                detail = "auto"
                            }
                        }
                    }
                }
        },
        max_tokens = maxTokens,
        temperature = temperature
    };
    return payload;
}