Skip to content

Latest commit

 

History

History
29 lines (26 loc) · 2.59 KB

File metadata and controls

29 lines (26 loc) · 2.59 KB

IPEX-LLM Examples on Intel NPU

This folder contains examples of running IPEX-LLM on Intel NPU:

  • LLM: examples of running large language models using IPEX-LLM optimizations
  • Multimodal: examples of running large multimodal models using IPEX-LLM optimizations

Verified Models on Intel NPU

Model Model Link
Llama2 meta-llama/Llama-2-7b-chat-hf
Llama3 meta-llama/Meta-Llama-3-8B-Instruct
Llama3.2-1B meta-llama/Llama-3.2-1B-Instruct
Llama3.2-3B meta-llama/Llama-3.2-3B-Instruct
Chatglm3 THUDM/chatglm3-6b
Chatglm2 THUDM/chatglm2-6b
Qwen2 Qwen/Qwen2-7B-Instruct, Qwen/Qwen2-1.5B-Instruct
Qwen2.5 Qwen/Qwen2.5-7B-Instruct
MiniCPM openbmb/MiniCPM-2B-sft-bf16
Phi-3 microsoft/Phi-3-mini-4k-instruct
Stablelm stabilityai/stablelm-zephyr-3b
Baichuan2 baichuan-inc/Baichuan2-7B-Chat
Deepseek deepseek-ai/deepseek-coder-6.7b-instruct
Mistral mistralai/Mistral-7B-Instruct-v0.1
Phi-3-Vision microsoft/Phi-3-vision-128k-instruct
MiniCPM-Llama3-V-2_5 openbmb/MiniCPM-Llama3-V-2_5
MiniCPM-V-2_6 openbmb/MiniCPM-V-2_6
Bce-Embedding-Base-V1 maidalun1020/bce-embedding-base_v1
Speech_Paraformer-Large iic/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-pytorch