Skip to content

Commit 935c134

Browse files
committed
gpu memory limitation on vllm container
1 parent 3cfb835 commit 935c134

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

docker-compose.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@ x-vllm: &vllm_template
4444
command: >
4545
--model ${EMBEDDER_MODEL_NAME:-jinaai/jina-embeddings-v3}
4646
--trust-remote-code
47+
--gpu_memory_utilization 0.3
4748
ports:
4849
- ${VLLM_PORT:-8000}:8000
4950

0 commit comments

Comments
 (0)