TypeError: bad operand type for unary -: 'NoneType'

[wln20]

An error occurred when I tried to run Mistral-7b-instruct-v0.2 model:

...
  File "~/anaconda/envs/streamingllm/lib/python3.9/site-packages/transformers/models/mistral/modeling_mistral.py", line 88, in _make_sliding_window_causal_mask
    mask = torch.triu(mask, diagonal=-sliding_window)
TypeError: bad operand type for unary -: 'NoneType'

I inspected the position causing the error and found the sliding_window is None here. I noticed that the problem raised because the model.model.config.sliding_window attribution is None and doesn't get the correct value. Therefore I manually set it after the instantiation of the model, and the code seems to run correctly:

sliding_window = 252
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    # for efficiency:
    device_map="auto",
    torch_dtype=torch.float16,
    # `attention_sinks`-specific arguments:
    attention_sink_size=4,
    attention_sink_window_size=sliding_window, # <- Low for the sake of faster generation
)
model.model.config.sliding_window = sliding_window