Add llamafile-convert command

[chand1012]

This adds a simple command for converting GGUF files and URLs to GGUF files to llamafiles quickly and with low friction. Usage as follows:

llamafile-convert <gguf path or url> [cli|server|both]

If the second argument is not specified, both is assumed. Tested on macOS and Linux, not tested on Windows.

[jart]

Thanks for sending this Chandler! I notice this script has already gotten four upvotes. So I think the convenient interface you've designed is worth having. I've left a lot of comments, with the intention of providing you with the best advice possible for a program that's destined to be installed on a large number of machines.

[jart]

Here's the POSIX builtin syntax for doing this:

SCRIPTNAME=${0##*/}

[jart]

Please use /bin/sh. If you're using Linux then most systems should have that configured to be the Almquist Shell, which is a minimal POSIX compliant shell that's ultra fast.

[jart]

I think glob equals is a bash extension. The POSIX way to do this would be:

if [ x"$FILE" != x"${FILE#http*}" ]; then  # FILE.startswith("http")

Which tests that $FILE is not equal to itself stripped of the http prefix.

Even though spaces technically aren't allowed in URLs, it's safer in general to have the quotes here, since it's user input. Many would view that as good from a security standpoint too.

[jart]

Some stock systems don't have wget but have curl. With other systems, it's the other way around. I recommend copying and pasting the downloader detection code I wrote in this script:

llamafile/build/download-cosmocc.sh

Lines 55 to 65 in 586b408

	if WGET=$(command -v wget 2>/dev/null); then
	DOWNLOAD=$WGET
	DOWNLOAD_ARGS=-O
	elif CURL=$(command -v curl 2>/dev/null); then
	DOWNLOAD=$CURL
	DOWNLOAD_ARGS=-fLo
	else
	printf '%s\n' "$0: fatal error: you need to install either wget or curl" >&2
	printf '%s\n' "please download https://cosmo.zip/pub/cosmos/bin/wget and put it on the system path" >&2
	abort
	fi

You can then say:

"${DOWNLOAD}" ${DOWNLOAD_ARGS} "$FILENAME" "$FILE"

[jart]

This line should end with >&2 so it gets printed to standard error.

[jart]

Please use command -v instead of which because the prior is a POSIX-compliant shell builtin that's faster.

[jart]

You can do this using builtins as follows:

CLI_ARGS="-m
$FILE
...
"

Then, rather than echo'ing it later on, do this:

printf %s "$CLI_ARGS" >.args

[jart]

This comment doesn't add value. Comments should be used to explain things that aren't obvious from reading the code.

[jart]

Some environments theoretically map rm to rm -i so it asks an interactive question before destroying data. To work around that, it's recommended that rm -f be favored in scripts.

[jart]

It's important to check the return code of processes launched by the script. For example, here, you could say convert_to_cli || abort where abort could be defined as a shell function similar to my script here:

llamafile/build/download-cosmocc.sh

Lines 14 to 17 in 586b408

	abort() {
	printf '%s\n' "download terminated." >&2
	exit 1
	}

[jart]

Thanks again Chandler. Quick drive by review.

[jart]

Could you remove the [cli|server|both] argument? The llamafile and llamafile-server commands are now unified at head. It's no longer needed.

[chand1012]

New usage as follows.

llamafile-convert <gguf path or url>

[jart]

Looks good. Thank you.

One thing you might want to do, is have an optional [-o OUTPUT] flag for overriding the generated pathname. If you want to code it, feel free to do so in a follow-up PR.

[stlhood]

Thank you @chand1012 for this very useful contribution!

[ProtoBelisarius]

Is there a way to use convert for multimodal models like llava? Because those not only need the .gguf but also the mmproj model. How do I include both models? ShareGPT4V is better than llava and works the same with llama.cpp so it should work.
This is an example command to run it with llama.cpp if you dont believe me:
./server -m models/ShareGPT4V-7B_Q5_K_M.gguf -t 6 -c 4096 --mmproj models/mmproj-model-f16.gguf