How to send many messages to open AI endpoint in one request?

[Stealthwriter]

Hi,

I have a model that extracts sentences from Text. I want to send many pieces of Text to the endpoint in a batch.

in Open ai they don't support batch requests.

I don't want to send many single requests.

How can I send a batch requests to /generate but still use the chat template?

[merrymercy]

You can apply the chat template by yourself in your client code.