[Bug]: opt rtn throws exepction at 3bits

[wenhuach21]

Problem Description

(autoround) wenhuach@mlp-dgx-01:~/auto-round-main$ CUDA_VISIBLE_DEVICES=6 python3 -m auto_round /models/opt-125m --tasks lambada_openai --iters 0 --scheme W3A16
2026-07-10 16:47:36 INFO main.py L293: start to quantize /models/opt-125m
2026-07-10 16:47:36 INFO config.py L53: enable_opt_rtn is turned on, set --disable_opt_rtn for higher speed at the cost of accuracy.
Loading weights: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 196/196 [00:00<00:00, 3916.25it/s]
[transformers] loss_type=None was set in the config but it is unrecognized. Using the default loss: ForCausalLMLoss.
2026-07-10 16:47:38 INFO base.py L767: 'enable_torch_compile' is set to False by default. Enabling it can reduce tuning cost by 20%, but it might throw an exception.
2026-07-10 16:47:38 INFO data_driven.py L1216: start to compute imatrix
2026-07-10 16:47:38 INFO calib_dataset.py L977: Preprocessing calibration dataset in a subprocess to avoid memory leaks...
2026-07-10 16:47:50 INFO calib_dataset.py L977: Preprocessing calibration dataset in a subprocess to avoid memory leaks...
Quantizing model.decoder.layers.0: 0%| | 0/12 [00:02<?, ?it/s]2026-07-10 16:48:08 INFO device.py L1450: 'peak_ram': 2.02GB, 'peak_vram': 1.5GB
Quantizing model.decoder.layers.1: 8%|█████████████████▍ | 1/12 [00:07<01:25, 7.74s/it]2026-07-10 16:48:09 WARNING logging.py L340: please use more data via setting nsamples to improve accuracy as calibration activations contain 0
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/home/wenhuach/auto-round-main/auto_round/main.py", line 20, in
run()
~~~^^
File "/home/wenhuach/auto-round-main/auto_round/cli/main.py", line 467, in run
start(argv=command_argv)
~~~~~^^^^^^^^^^^^^^^^^^^
File "/home/wenhuach/auto-round-main/auto_round/cli/main.py", line 235, in start
tune(args)
~~~~^^^^^^
File "/home/wenhuach/auto-round-main/auto_round/cli/main.py", line 370, in tune
model, folders = autoround.quantize_and_save(args.output_dir, format=args.format) # pylint: disable=no-member
~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/wenhuach/auto-round-main/auto_round/compressors/base.py", line 1732, in quantize_and_save
self.quantize()
~~~~~~~~~~~~~^^
File "/home/wenhuach/auto-round-main/auto_round/compressors/data_driven.py", line 1278, in quantize
return self._quantize_impl()
~~~~~~~~~~~~~~~~~~~^^
File "/home/wenhuach/miniforge3/envs/autoround/lib/python3.13/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
return func(*args, **kwargs)
File "/home/wenhuach/auto-round-main/auto_round/compressors/data_driven.py", line 1304, in _quantize_impl
self._quant_rtn_with_imatrix()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/home/wenhuach/auto-round-main/auto_round/compressors/data_driven.py", line 1235, in _quant_rtn_with_imatrix
self._quantize_via_rtn_blockwise()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/home/wenhuach/auto-round-main/auto_round/compressors/data_driven.py", line 1178, in _quantize_via_rtn_blockwise
self.quantizer.quantize_block(ctx)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^
File "/home/wenhuach/miniforge3/envs/autoround/lib/python3.13/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
return func(*args, **kwargs)
File "/home/wenhuach/auto-round-main/auto_round/algorithms/quantization/rtn/quantizer.py", line 162, in quantize_block
self.quantize_layer_outside_block(m.global_name)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
File "/home/wenhuach/auto-round-main/auto_round/algorithms/quantization/base.py", line 135, in quantize_layer_outside_block
self.quantize_layer_via_rtn(layer_name, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
File "/home/wenhuach/miniforge3/envs/autoround/lib/python3.13/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
return func(*args, **kwargs)
File "/home/wenhuach/auto-round-main/auto_round/algorithms/quantization/base.py", line 86, in quantize_layer_via_rtn
layer = layer.unwrapper({})
File "/home/wenhuach/auto-round-main/auto_round/wrapper.py", line 363, in unwrapper
qdq_weight, scale, zp = self._qdq_weight(v, min_scale, max_scale)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/wenhuach/auto-round-main/auto_round/wrapper.py", line 268, in _qdq_weight
weight_q, scale, zp = self.weight_quant_func(
~~~~~~~~~~~~~~~~~~~~~~^
weight.to(self.device),
^^^^^^^^^^^^^^^^^^^^^^^
...<13 lines>...
**quant_kwargs,
^^^^^^^^^^^^^^^
)
^
File "/home/wenhuach/auto-round-main/auto_round/data_type/int.py", line 115, in quant_tensor_opt_rtn_sym
imatrix = _imatrix_handle_zero(imatrix, tensor, bits)
File "/home/wenhuach/auto-round-main/auto_round/data_type/gguf.py", line 405, in _imatrix_handle_zero
tmp_quant_weights = tmp_quant_weights.to(imatrix.dtype)
^^^^^^^^^^^^^^^^^
UnboundLocalError: cannot access local variable 'tmp_quant_weights' where it is not associated with a value
Quantizing model.decoder.layers.1: 8%|█████████████████▍

Reproduction Steps

~

Environment Information

~

Error Logs

Additional Context

No response

[wenhuach21]

#2039