feat: supported multi-modal embeddings from Azure AI Vision service

[adubovik]

Introduced a new env var: AZURE_AI_VISION_DEPLOYMENTS that contains a comma-separated list of Azure AI Vision embedding deployments.

The upstreams of such deployments configured in the DIAL Core config are expected to point to Azure AI Vision service: https://<azure-ai-vision-service-name>.cognitiveservices.azure.com/

Under the hood the adapter calls /retrieval:vectorizeImage and /retrieval:vectorizeText endpoints of the Azure AI Vision service to compute embeddings for images and texts.

Example configuration

Replace {adapter-openai-hostname} and {azure-ai-vision-service-name} with values relevant for our setup.

DIAL Core config

{
    "models": {
        "my-azure-ai-vision-embeddings": {
            "type": "embedding",
            "endpoint": "http://{adapter-openai-hostname}/openai/deployments/my-azure-ai-vision-embeddings/embeddings",
            "pricing": {
                "unit": "token",
                "prompt": "0.0009"
            },
            "upstreams": [
                {
                    "endpoint": "https://{azure-ai-vision-service-name}.cognitiveservices.azure.com"
                }
            ]
        }
    }
}

OpenAI adapter configuration

AZURE_AI_VISION_DEPLOYMENTS=my-azure-ai-vision-embeddings

[adubovik]

/deploy-review

Environment URL: https://chat-ai-dial-adapter-openai-pr-162.nightly-test.deltixhub.io
E2E tests status: failed

[adubovik]

/deploy-review

• Deployment failed: failed

[adubovik]

/deploy-review

Environment URL: https://chat-ai-dial-adapter-openai-pr-162.nightly-test.deltixhub.io
E2E tests status: success