AVX / AVX512 / SSE3 optimizations not detected

[tye-singwa]

Hi!
I've tried to install python package, but seems that AVX / AVX2 / SSE3 optimizations has been not detected, as per codewars/runner#118 (comment) and per makefile ggml-org/llama.cpp@872c365#diff-76ed074a9305c04054cdebb9e9aad2d818052b07091de1f20cad0bbac34ffb52R79-R82 it is not always enabled

Also i see cmake build uses makefile https://github.com/abetlen/llama-cpp-python/blob/main/CMakeLists.txt#L8, maybe its possible to change it back?

llama.cpp weights detected: models/vicuna-AlekseyKorshuk-7B-GPTQ-4bit-128g-GGML/ggml-vicuna-AlekseyKorshuk-7B-GPTQ-4bit-128g.bin

llama.cpp: loading model from models/vicuna-AlekseyKorshuk-7B-GPTQ-4bit-128g-GGML/ggml-vicuna-AlekseyKorshuk-7B-GPTQ-4bit-128g.bin
llama_model_load_internal: format     = ggjt v1 (latest)
llama_model_load_internal: n_vocab    = 32001
llama_model_load_internal: n_ctx      = 2048
llama_model_load_internal: n_embd     = 4096
llama_model_load_internal: n_mult     = 256
llama_model_load_internal: n_head     = 32
llama_model_load_internal: n_layer    = 32
llama_model_load_internal: n_rot      = 128
llama_model_load_internal: ftype      = 4 (mostly Q4_1, some F16)
llama_model_load_internal: n_ff       = 11008
llama_model_load_internal: n_parts    = 1
llama_model_load_internal: model size = 7B
llama_model_load_internal: ggml ctx size =  59.11 KB
llama_model_load_internal: mem required  = 6925.09 MB (+ 1026.00 MB per state)
llama_init_from_file: kv self size  = 1024.00 MB
AVX = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 0 | VSX = 0 |

Thanks

[abetlen]

@tye-singwa just pushed a new version to PyPI with a build flag that lets you force cmake installation, you can use FORCE_CMAKE=1 pip install --upgrade llama-cpp-python

[tye-singwa]

Thanks! It works now

AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 |

[ZacharyDK]

Having the same issue. AVX still not detected....