Support PyTorch set_per_process_memory_fraction
#39
Labels
Comments
kcm
added
enhancement
|
One use case is for AWS vGPU support so that multiple consumers of the vGPU device(s) don't assume they have exclusive rights to the full resource usage. |
gauthamchandra
added a commit
to uhadmin/weaviate-helm
that referenced
this issue
Dec 29, 2022
Due to the lack of support for GPU virtualization by Weaviate (see weaviate/t2v-transformers-models#39 for details), we need to ensure each pod that uses the GPU, utilizes the full pod without sharing it. The way to ensure this is to schedule each GPU enabled pod to its own dedicated node using Kubernetes's anti-affinity feature. This addresses this to make it easy to run clusters with GPU support.
gauthamchandra
added a commit
to uhadmin/weaviate-helm
that referenced
this issue
Dec 29, 2022
Due to the lack of support for GPU virtualization by Weaviate (see weaviate/t2v-transformers-models#39 for details), we need to ensure each pod that uses the GPU, utilizes the full pod without sharing it. The way to ensure this is to schedule each GPU enabled pod to its own dedicated node using Kubernetes's anti-affinity feature. This addresses this to make it easy to run clusters with GPU support.
gauthamchandra
added a commit
to uhadmin/weaviate-helm
that referenced
this issue
Dec 29, 2022
Due to the lack of support for GPU virtualization by Weaviate (see weaviate/t2v-transformers-models#39 for details), we need to ensure each pod that uses the GPU, utilizes the full pod without sharing it. The way to ensure this is to schedule each GPU enabled pod to its own dedicated node using Kubernetes's anti-affinity feature. This addresses this to make it easy to run clusters with GPU support.
gauthamchandra
added a commit
to uhadmin/weaviate-helm
that referenced
this issue
Dec 29, 2022
Due to the lack of support for GPU virtualization by Weaviate (see weaviate/t2v-transformers-models#39 for details), we need to ensure each pod that uses the GPU, utilizes the full pod without sharing it. The way to ensure this is to schedule each GPU enabled pod to its own dedicated node using Kubernetes's anti-affinity feature. This addresses this to make it easy to run clusters with GPU support.
gauthamchandra
added a commit
to uhadmin/weaviate-helm
that referenced
this issue
Apr 3, 2023
Due to the lack of support for GPU virtualization by Weaviate (see weaviate/t2v-transformers-models#39 for details), we need to ensure each pod that uses the GPU, utilizes the full pod without sharing it. The way to ensure this is to schedule each GPU enabled pod to its own dedicated node using Kubernetes's anti-affinity feature. This addresses this to make it easy to run clusters with GPU support.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Summary
PyTorch allows a limit for GPU memory. This is useful, for example, when a GPU resource is shared.
Proposal
This setting takes a percentage [0-1] and a device (optional). Use an environment variable alongside
ENABLE_CUDAof the formatCUDA_MEMORY_FRACTIONwhere the value is 0.0-1.0 and passed tofraction. Additionally, if set, check and preferCUDA_MEMORY_FRACTION_...variable(s), where the value is the same format, and the...is passed todevicefor each variable found.Questions
CUDA_MEMORY_FRACTION/CUDA_MEMORY_FRACTION_...?The text was updated successfully, but these errors were encountered: