Skip to content

Commit

Permalink
Update README.md (#1570)
Browse files Browse the repository at this point in the history
Signed-off-by: Dong, Bo <[email protected]>
  • Loading branch information
a32543254 committed May 24, 2024
1 parent 0dcee38 commit 93b12e9
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,7 @@ inputs = tokenizer(prompt, return_tensors="pt").input_ids
model = AutoModelForCausalLM.from_pretrained(model_name, load_in_4bit=True)
outputs = model.generate(inputs)
```
You can also load GGUF format model from Huggingface, and we will use [NeuralSpeed](https://github.com/intel/neural-speed) to accelerate the inference on CPUs.
You can also load GGUF format model from Huggingface, we only support Q4_0 gguf format for now.
```python
from transformers import AutoTokenizer
from intel_extension_for_transformers.transformers import AutoModelForCausalLM
Expand Down

0 comments on commit 93b12e9

Please sign in to comment.