You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've been trying with different LLMs but I haven't been able to make it works. Could you bring some light?
luispoveda93@LUIS-PC:~/mlmm-evaluation$ bash scripts/run.sh es microsoft/Phi-3-mini-4k-instruct
Selected Tasks: ['arc_es', 'hellaswag_es', 'mmlu_es']
config.json: 100%|█████████████████████████████████████████████████████████████████████| 904/904 [00:00<00:00, 9.41MB/s]
Traceback (most recent call last):
File "/home/luispoveda93/mlmm-evaluation/main.py", line 135, in<module>main()
File "/home/luispoveda93/mlmm-evaluation/main.py", line 108, in main
results = evaluator.open_llm_evaluate(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/luispoveda93/mlmm-evaluation/lm_eval/utils.py", line 205, in _wrapper
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/home/luispoveda93/mlmm-evaluation/lm_eval/evaluator.py", line 66, in open_llm_evaluate
lm = lm_eval.models.get_model(model).create_from_arg_string(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/luispoveda93/mlmm-evaluation/lm_eval/base.py", line 116, in create_from_arg_string
return cls(**args, **args2)
^^^^^^^^^^^^^^^^^^^^
File "/home/luispoveda93/mlmm-evaluation/lm_eval/models/huggingface.py", line 169, in __init__
self._config = self.AUTO_CONFIG_CLASS.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/luispoveda93/.pyenv/versions/3.11.9/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 931, in from_pretrained
trust_remote_code = resolve_trust_remote_code(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/luispoveda93/.pyenv/versions/3.11.9/lib/python3.11/site-packages/transformers/dynamic_module_utils.py", line 627, in resolve_trust_remote_code
raise ValueError(
ValueError: Loading microsoft/Phi-3-mini-4k-instruct requires you to execute the configuration file in that repo on your local machine. Make sure you have read the code there to avoid malicious use, thenset the option `trust_remote_code=True` to remove this error.
The text was updated successfully, but these errors were encountered:
Please try logging into your Hugging Face account before running our script. In your terminal, please run huggingFace-cli login, then enter your HF read token, and after logging in, try to run the script above again.
Hello,
I've been trying with different LLMs but I haven't been able to make it works. Could you bring some light?
The text was updated successfully, but these errors were encountered: