@@ -170,19 +170,24 @@ ChatCompletionMessage(content='There once was a programmer named Mike\nWho wrote
170170We also provide example llamafiles for other models, so you can easily
171171try out llamafile with different kinds of LLMs.
172172
173- | Model | Size | License | llamafile | other quants |
174- | --- | --- | --- | --- | --- |
175- | LLaVA 1.5 | 3.97 GB | [ LLaMA 2] ( https://ai.meta.com/resources/models-and-libraries/llama-downloads/ ) | [ llava-v1.5-7b-q4.llamafile] ( https://huggingface.co/Mozilla/llava-v1.5-7b-llamafile/resolve/main/llava-v1.5-7b-q4.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/llava-v1.5-7b-llamafile ) |
176- | TinyLlama-1.1B | 2.05 GB | [ Apache 2.0] ( https://choosealicense.com/licenses/apache-2.0/ ) | [ TinyLlama-1.1B-Chat-v1.0.F16.llamafile] ( https://huggingface.co/Mozilla/TinyLlama-1.1B-Chat-v1.0-llamafile/resolve/main/TinyLlama-1.1B-Chat-v1.0.F16.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/TinyLlama-1.1B-Chat-v1.0-llamafile ) |
177- | Mistral-7B-Instruct | 3.85 GB | [ Apache 2.0] ( https://choosealicense.com/licenses/apache-2.0/ ) | [ mistral-7b-instruct-v0.2.Q4\_ 0.llamafile] ( https://huggingface.co/Mozilla/Mistral-7B-Instruct-v0.2-llamafile/resolve/main/mistral-7b-instruct-v0.2.Q4_0.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/Mistral-7B-Instruct-v0.2-llamafile ) |
178- | Phi-3-mini-4k-instruct | 7.67 GB | [ Apache 2.0] ( https://huggingface.co/Mozilla/Phi-3-mini-4k-instruct-llamafile/blob/main/LICENSE ) | [ Phi-3-mini-4k-instruct.F16.llamafile] ( https://huggingface.co/Mozilla/Phi-3-mini-4k-instruct-llamafile/resolve/main/Phi-3-mini-4k-instruct.F16.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/Phi-3-mini-4k-instruct-llamafile ) |
179- | Mixtral-8x7B-Instruct | 30.03 GB | [ Apache 2.0] ( https://choosealicense.com/licenses/apache-2.0/ ) | [ mixtral-8x7b-instruct-v0.1.Q5\_ K\_ M.llamafile] ( https://huggingface.co/Mozilla/Mixtral-8x7B-Instruct-v0.1-llamafile/resolve/main/mixtral-8x7b-instruct-v0.1.Q5_K_M.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/Mixtral-8x7B-Instruct-v0.1-llamafile ) |
180- | WizardCoder-Python-34B | 22.23 GB | [ LLaMA 2] ( https://ai.meta.com/resources/models-and-libraries/llama-downloads/ ) | [ wizardcoder-python-34b-v1.0.Q5\_ K\_ M.llamafile] ( https://huggingface.co/Mozilla/WizardCoder-Python-34B-V1.0-llamafile/resolve/main/wizardcoder-python-34b-v1.0.Q5_K_M.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/WizardCoder-Python-34B-V1.0-llamafile ) |
181- | WizardCoder-Python-13B | 7.33 GB | [ LLaMA 2] ( https://ai.meta.com/resources/models-and-libraries/llama-downloads/ ) | [ wizardcoder-python-13b.llamafile] ( https://huggingface.co/jartine/wizardcoder-13b-python/resolve/main/wizardcoder-python-13b.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/jartine/wizardcoder-13b-python ) |
182- | LLaMA-3-Instruct-70B | 37.25 GB | [ llama3] ( https://huggingface.co/Mozilla/Meta-Llama-3-8B-Instruct-llamafile/blob/main/Meta-Llama-3-Community-License-Agreement.txt ) | [ Meta-Llama-3-70B-Instruct.Q4\_ 0.llamafile] ( https://huggingface.co/Mozilla/Meta-Llama-3-70B-Instruct-llamafile/resolve/main/Meta-Llama-3-70B-Instruct.Q4_0.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/Meta-Llama-3-70B-Instruct-llamafile ) |
183- | LLaMA-3-Instruct-8B | 5.37 GB | [ llama3] ( https://huggingface.co/Mozilla/Meta-Llama-3-8B-Instruct-llamafile/blob/main/Meta-Llama-3-Community-License-Agreement.txt ) | [ Meta-Llama-3-8B-Instruct.Q5\_ K\_ M.llamafile] ( https://huggingface.co/Mozilla/Meta-Llama-3-8B-Instruct-llamafile/resolve/main/Meta-Llama-3-8B-Instruct.Q5_K_M.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/Meta-Llama-3-8B-Instruct-llamafile ) |
184- | Rocket-3B | 1.89 GB | [ cc-by-sa-4.0] ( https://creativecommons.org/licenses/by-sa/4.0/deed.en ) | [ rocket-3b.Q5\_ K\_ M.llamafile] ( https://huggingface.co/Mozilla/rocket-3B-llamafile/resolve/main/rocket-3b.Q5_K_M.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/rocket-3B-llamafile ) |
185- | OLMo-7B | 5.68 GB | [ Apache 2.0] ( https://huggingface.co/Mozilla/OLMo-7B-0424-llamafile/blob/main/LICENSE ) | [ OLMo-7B-0424.Q6\_ K.llamafile] ( https://huggingface.co/Mozilla/OLMo-7B-0424-llamafile/resolve/main/OLMo-7B-0424.Q6_K.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/OLMo-7B-0424-llamafile ) |
173+ | Model | Size | License | llamafile | other quants |
174+ | --- | --- | --- | --- | --- |
175+ | LLaMA 3.2 3B Instruct | 2.62 GB | [ LLaMA 3.2] ( https://huggingface.co/Mozilla/Llama-3.2-3B-Instruct-llamafile/blob/main/LICENSE ) | [ Llama-3.2-3B-Instruct.Q6\_ K.llamafile] ( https://huggingface.co/Mozilla/Llama-3.2-3B-Instruct-llamafile/blob/main/Llama-3.2-3B-Instruct.Q6_K.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/Llama-3.2-3B-Instruct-llamafile ) |
176+ | LLaMA 3.2 1B Instruct | 1.11 GB | [ LLaMA 3.2] ( https://huggingface.co/Mozilla/Llama-3.2-1B-Instruct-llamafile/blob/main/LICENSE ) | [ Llama-3.2-1B-Instruct.Q6\_ K.llamafile] ( https://huggingface.co/Mozilla/Llama-3.2-1B-Instruct-llamafile/blob/main/Llama-3.2-1B-Instruct.Q6_K.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/Llama-3.2-1B-Instruct-llamafile ) |
177+ | Gemma 2 2B Instruct | 2.32 GB | [ Gemma 2] ( https://huggingface.co/Mozilla/gemma-2-2b-it-llamafile/blob/main/LICENSE ) | [ gemma-2-2b-it.Q6\_ K.llamafile] ( https://huggingface.co/Mozilla/gemma-2-2b-it-llamafile/blob/main/gemma-2-2b-it.Q6_K.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/gemma-2-2b-it-llamafile ) |
178+ | Gemma 2 9B Instruct | 7.76 GB | [ Gemma 2] ( https://huggingface.co/Mozilla/gemma-2-9b-it-llamafile/blob/main/LICENSE ) | [ gemma-2-9b-it.Q6\_ K.llamafile] ( https://huggingface.co/Mozilla/gemma-2-9b-it-llamafile/blob/main/gemma-2-9b-it.Q6_K.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/gemma-2-9b-it-llamafile ) |
179+ | Gemma 2 27B Instruct | 22.5 GB | [ Gemma 2] ( https://huggingface.co/Mozilla/gemma-2-27b-it-llamafile/blob/main/LICENSE ) | [ gemma-2-27b-it.Q6\_ K.llamafile] ( https://huggingface.co/Mozilla/gemma-2-27b-it-llamafile/blob/main/gemma-2-27b-it.Q6_K.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/gemma-2-27b-it-llamafile ) |
180+ | LLaVA 1.5 | 3.97 GB | [ LLaMA 2] ( https://ai.meta.com/resources/models-and-libraries/llama-downloads/ ) | [ llava-v1.5-7b-q4.llamafile] ( https://huggingface.co/Mozilla/llava-v1.5-7b-llamafile/resolve/main/llava-v1.5-7b-q4.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/llava-v1.5-7b-llamafile ) |
181+ | TinyLlama-1.1B | 2.05 GB | [ Apache 2.0] ( https://choosealicense.com/licenses/apache-2.0/ ) | [ TinyLlama-1.1B-Chat-v1.0.F16.llamafile] ( https://huggingface.co/Mozilla/TinyLlama-1.1B-Chat-v1.0-llamafile/resolve/main/TinyLlama-1.1B-Chat-v1.0.F16.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/TinyLlama-1.1B-Chat-v1.0-llamafile ) |
182+ | Mistral-7B-Instruct | 3.85 GB | [ Apache 2.0] ( https://choosealicense.com/licenses/apache-2.0/ ) | [ mistral-7b-instruct-v0.2.Q4\_ 0.llamafile] ( https://huggingface.co/Mozilla/Mistral-7B-Instruct-v0.2-llamafile/resolve/main/mistral-7b-instruct-v0.2.Q4_0.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/Mistral-7B-Instruct-v0.2-llamafile ) |
183+ | Phi-3-mini-4k-instruct | 7.67 GB | [ Apache 2.0] ( https://huggingface.co/Mozilla/Phi-3-mini-4k-instruct-llamafile/blob/main/LICENSE ) | [ Phi-3-mini-4k-instruct.F16.llamafile] ( https://huggingface.co/Mozilla/Phi-3-mini-4k-instruct-llamafile/resolve/main/Phi-3-mini-4k-instruct.F16.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/Phi-3-mini-4k-instruct-llamafile ) |
184+ | Mixtral-8x7B-Instruct | 30.03 GB | [ Apache 2.0] ( https://choosealicense.com/licenses/apache-2.0/ ) | [ mixtral-8x7b-instruct-v0.1.Q5\_ K\_ M.llamafile] ( https://huggingface.co/Mozilla/Mixtral-8x7B-Instruct-v0.1-llamafile/resolve/main/mixtral-8x7b-instruct-v0.1.Q5_K_M.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/Mixtral-8x7B-Instruct-v0.1-llamafile ) |
185+ | WizardCoder-Python-34B | 22.23 GB | [ LLaMA 2] ( https://ai.meta.com/resources/models-and-libraries/llama-downloads/ ) | [ wizardcoder-python-34b-v1.0.Q5\_ K\_ M.llamafile] ( https://huggingface.co/Mozilla/WizardCoder-Python-34B-V1.0-llamafile/resolve/main/wizardcoder-python-34b-v1.0.Q5_K_M.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/WizardCoder-Python-34B-V1.0-llamafile ) |
186+ | WizardCoder-Python-13B | 7.33 GB | [ LLaMA 2] ( https://ai.meta.com/resources/models-and-libraries/llama-downloads/ ) | [ wizardcoder-python-13b.llamafile] ( https://huggingface.co/jartine/wizardcoder-13b-python/resolve/main/wizardcoder-python-13b.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/jartine/wizardcoder-13b-python ) |
187+ | LLaMA-3-Instruct-70B | 37.25 GB | [ llama3] ( https://huggingface.co/Mozilla/Meta-Llama-3-8B-Instruct-llamafile/blob/main/Meta-Llama-3-Community-License-Agreement.txt ) | [ Meta-Llama-3-70B-Instruct.Q4\_ 0.llamafile] ( https://huggingface.co/Mozilla/Meta-Llama-3-70B-Instruct-llamafile/resolve/main/Meta-Llama-3-70B-Instruct.Q4_0.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/Meta-Llama-3-70B-Instruct-llamafile ) |
188+ | LLaMA-3-Instruct-8B | 5.37 GB | [ llama3] ( https://huggingface.co/Mozilla/Meta-Llama-3-8B-Instruct-llamafile/blob/main/Meta-Llama-3-Community-License-Agreement.txt ) | [ Meta-Llama-3-8B-Instruct.Q5\_ K\_ M.llamafile] ( https://huggingface.co/Mozilla/Meta-Llama-3-8B-Instruct-llamafile/resolve/main/Meta-Llama-3-8B-Instruct.Q5_K_M.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/Meta-Llama-3-8B-Instruct-llamafile ) |
189+ | Rocket-3B | 1.89 GB | [ cc-by-sa-4.0] ( https://creativecommons.org/licenses/by-sa/4.0/deed.en ) | [ rocket-3b.Q5\_ K\_ M.llamafile] ( https://huggingface.co/Mozilla/rocket-3B-llamafile/resolve/main/rocket-3b.Q5_K_M.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/rocket-3B-llamafile ) |
190+ | OLMo-7B | 5.68 GB | [ Apache 2.0] ( https://huggingface.co/Mozilla/OLMo-7B-0424-llamafile/blob/main/LICENSE ) | [ OLMo-7B-0424.Q6\_ K.llamafile] ( https://huggingface.co/Mozilla/OLMo-7B-0424-llamafile/resolve/main/OLMo-7B-0424.Q6_K.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/OLMo-7B-0424-llamafile ) |
186191| * Text Embedding Models* | | | | |
187192| E5-Mistral-7B-Instruct | 5.16 GB | [ MIT] ( https://choosealicense.com/licenses/mit/ ) | [ e5-mistral-7b-instruct-Q5_K_M.llamafile] ( https://huggingface.co/Mozilla/e5-mistral-7b-instruct/resolve/main/e5-mistral-7b-instruct-Q5_K_M.llamafile?download=true ) | [ See HF repo] ( https://huggingface.co/Mozilla/e5-mistral-7b-instruct ) |
188193| mxbai-embed-large-v1 | 0.7 GB | [ Apache 2.0] ( https://choosealicense.com/licenses/apache-2.0/ ) | [ mxbai-embed-large-v1-f16.llamafile] ( https://huggingface.co/Mozilla/mxbai-embed-large-v1-llamafile/resolve/main/mxbai-embed-large-v1-f16.llamafile?download=true ) | [ See HF Repo] ( https://huggingface.co/Mozilla/mxbai-embed-large-v1-llamafile ) |
0 commit comments