Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When quantizing baichuan2-7B, I encountered an error: AttributeError: 'RMSNorm' object has no attribute 'epsilon'. #20

Open
Flyipig opened this issue Apr 24, 2024 · 0 comments

Comments

@Flyipig
Copy link

Flyipig commented Apr 24, 2024

Here are the details of the traceback:
Traceback (most recent call last):
File "/data/run01/scz0oo5/AutoSmoothQuant/autosmoothquant/examples/smoothquant_model.py", line 70, in
main()
File "/HOME/scz0oo5/.conda/envs/AUTOSmooth/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/data/run01/scz0oo5/AutoSmoothQuant/autosmoothquant/examples/smoothquant_model.py", line 65, in main
int8_model = quant_model_class.from_float(model, decoder_layer_scales, quant_config)
File "/data/run01/scz0oo5/AutoSmoothQuant/autosmoothquant/models/baichuan.py", line 561, in from_float
int8_module.model = Int8BaichuanModel.from_float(
File "/data/run01/scz0oo5/AutoSmoothQuant/autosmoothquant/models/baichuan.py", line 363, in from_float
int8_module.layers[i] = Int8BaichuanLayer.from_float(
File "/data/run01/scz0oo5/AutoSmoothQuant/autosmoothquant/models/baichuan.py", line 284, in from_float
int8_module.input_layernorm = Int8BaichuanRMSNorm.from_float(
File "/data/run01/scz0oo5/AutoSmoothQuant/autosmoothquant/models/baichuan.py", line 54, in from_float
int8_norm = Int8BaichuanRMSNorm(module.weight.numel(), module.epsilon)
File "/HOME/scz0oo5/.conda/envs/AUTOSmooth/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1688, in getattr
raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'")
AttributeError: 'RMSNorm' object has no attribute 'epsilon'

I guess this issue is related to the version of PyTorch being used. Could you please advise on the specific version of PyTorch required for using AUTOSmoothQuant?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant