You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I was trying to quantize an L3 8B model using a jupiter notebook I cooked up and I got this error:
"-- Resuming job",
"!! Note: Overriding options with settings from existing job",
"-- Input: /workspace/L3-8B-Lunar-Stheno",
"-- Output: /workspace/quants",
"-- Calibration dataset: /workspace/exllamav2/0000.parquet, 100 / 16 rows, 2048 tokens per sample\n",
"-- Target bits per weight: 5.5 (decoder), 6 (head)",
"-- Max shard size: 8192 MB",
"-- Token embeddings (measurement)...",
"Traceback (most recent call last):",
"File \"/workspace/exllamav2/convert.py\", line 1, in <module>",
"import exllamav2.conversion.convert_exl2",
"File \"/workspace/exllamav2/exllamav2/conversion/convert_exl2.py\", line 252, in <module>",
"embeddings(job, save_job, model)\n",
"File \"/workspace/exllamav2/exllamav2/conversion/measure.py\", line 81, in embeddings",
"module.load()",
"TypeError: _DecoratorContextManager.__call__() missing 1 required positional argument: 'orig_func'"
I have no clue what orig_func means, checked the docs, found nothing.
So umm could someone please help me fix this or at least tell me what orig_func means
The text was updated successfully, but these errors were encountered:
Masterjp123
changed the title
orig_funct Quantization error
orig_func Quantization error
Jul 25, 2024
The orig_func error relates to the @torch.inference_mode decorator used on that function, so something's screwy with either your Python or PyTorch version. What versions are you using?
I'm actually not sure if Torch 2.0.1 is still supported. I know there are wheels for it, and the wheel builds but I haven't tested it in a while. Since it's very old.
I was trying to quantize an L3 8B model using a jupiter notebook I cooked up and I got this error:
I have no clue what orig_func means, checked the docs, found nothing.
So umm could someone please help me fix this or at least tell me what orig_func means
The text was updated successfully, but these errors were encountered: