How to change model download location?

[aalyousfi]

Hi,

I know for Transformers, we can change the model download location with something like:

import os
os.environ['TRANSFORMERS_CACHE'] = '/home/ec2-user/hf_cache/'

But it seems to be not affecting AWQ model download. They are still being downloaded to Transformers' default location (~/.cache/huggingface/hub/).

I need to change the default location because I don't have enough space there. Is there a way to do that?

[rlouf]

Is that something that the AWQ library allows? If so could you pass this via model_kwargs?

[aalyousfi]

I think they all use the Transformers environment variable mentioned above (TRANSFORMERS_CACHE). I'm sure that Transformers library use it correctly because I tried it. But with Outlines and AWQ models there is something wrong. The models get downloaded to both the defaultt location (~/.cache/huggingface/hub/) and the location specified in the env variable. However, for the latter, the model is not fully downloaded, just a few megabytes worth of files are downloaded. I'm not sure what's the issue. I was hoping there was some configuration in Outlines that would allow us to choose model download location...

[rlouf]

It is very strange because we use HuggingFace and AWQ's model loading without modification. I'll give it a try. Can you confirm it otherwise works with transformers and awq using this environment variable with your current configuration?

[namin]

You can create a symlink to a directory elsewhere. For example:

cd ~/.cache/huggingface/
mv hub /home/ec2-user/hf_cache # if needed
ln -s /home/ec2-user/hf_cache hub

[aalyousfi]

Thank you very much, this solved the problem for me.