Production use

[paillave]

Hello,

You mention on the main page "this project is for educational purposes".

What makes it, according to you, not suitable for real world situations and production environments?

Thanks.

[BetaDoggo]

The main focus of this repo is running inference on LLaMa based models. LLaMa has a noncommercial license so it cannot be used beyond educational and research applications. This repo and ggml in general is also still under heavy development so it is not ideal for stable production environments.

[paillave]

So when you say "it cannot be used[...]", you mean "it legally cannot be used[...]"?

[paillave]

Actually, I just had a look at the license:

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished [...]

Unless this license is not legal because llama of facebook would have a licence that is not compatible, this can be used in production environment... if you like taking risks.

[BetaDoggo]

The license of this software allows commercial use but the license for the LLaMa model weights does not. llama.cpp also supports OpenLLaMa weights which do allow commercial use but the selection of models is much more limited.

[ianscrivener]

llama.cpp can run Open Llama which has an Apache-2.0 license
Facebook's Llama model does have a restrictive licence -

[ianscrivener]

an LLM just reminded me: "You can use software licensed under the Apache 2.0 license commercially. Under this license, end-users can create their own proprietary software and offer the licensed code of the original software to customers. End-users can utilize the Apache 2.0 license in any commercially licensed software or enterprise application for free. Companies can include the licensed code in proprietary software that they then sell to customers. You do not have an obligation to share your modified source code when using software licensed under the Apache 2.0 license."