Unable to serve too consecutive requests seldon-core + mlserver

[EmanuelOverflow]

I have a local seldon-core kind cluster. I followed the guide on seldon docs at [https://docs.seldon.io/projects/seldon-core/en/latest/install/kind.html#](Install Locally) page. The server is running. I deployed the application with mlserver using SeldonDeployment v2.

I'm testing it with port-forwarding over 0.0.0.0 to access it over a company VPN. I have issues when I try to send consecutive, non parallel, requests. Some requests are served other not. The requests sometime seems to arrive because it is printed on terminal, but it stuck indefinitely (I waited more than 10 minutes). I tried to to the request using Postman and a web app, but the result is the same. No error shown on seldon cluster neither on Postman. I saw that the payload, that contains an image base64 encoded, is uploaded as the uploading progress is printed, but at the end the request does not return. I'm unable to see any error on various logs. I'm using the kubernetes-dashboard to navigate over all components.

I tested the image locally with multiple parallel requests and it works

1. mlserver start ... [works]
2. mlserver build ...
3. docker run ... [works]

The problem seems to be relevant to the deploy on seldon core. Any suggestions? Can I set the verbose level to print more information on requests?

Environment

Local Machine - Windows 11 Pro
Docker Desktop 4.30 on WSL2
Kind v0.23.0 go1.21.10 windows/amd64
Cluster with seldon-control-panel and seldon-worker
Istio client version: 1.22.0
Istio control plane version: 1.22.0
Istio data plane version: 1.22.0 (7 proxies)
Seldon core have been installed with helm
Kubectl Client Version: v1.30.0
Kubectl Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Kubectl Server Version: v1.30.0

Configurations are in the attached files.
I made the docker image public. To test it use a json payload like:

{ "inputs":[ { "data":[ "\/9j\/4AAQSkZJRgABAQ......" ], "shape":[ 1, 1 ], "name":"images", "datatype":"BYTES", "parameters":{ "content_type":"base64" } } ] }

seldoncluster.txt
seldondeployment.txt

Logs of your model: just a lot of "INFO: 10.244.1.4:0 - "POST /v2/models/fruits-quality-inspector/infer HTTP/1.1" 200 OK"