Initial commit

.github/workflows/release.yml

@@ -0,0 +1,36 @@
+name: Release
+
+on:
+  push:
+    tags:
+      - '*'
+
+permissions:
+  contents: write
+
+env:
+  ZIP_NAME: jan-models-bggpt-${{ github.ref_name }}.zip
+
+jobs:
+  release:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Generate models
+        working-directory: utils/model-generator
+        run: |
+          pip install -r requirements.txt
+          python main.py
+
+      # Create a .zip file of the models directory without the parent directory
+      - name: Create .zip file
+        working-directory: models
+        run: |
+          zip -r ../${{ env.ZIP_NAME }} .
+
+      - name: Release
+        uses: softprops/action-gh-release@v1
+        with:
+          files: ${{ env.ZIP_NAME }}

.gitignore

@@ -0,0 +1,2 @@
+*.gguf
+models

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Ivaylo Stoyanov
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,53 @@
+# BgGPT for Jan 👋
+[![Release](https://github.com/ivkos/jan-models-bggpt/actions/workflows/release.yml/badge.svg)](https://github.com/ivkos/jan-models-bggpt/actions/workflows/release.yml)
+
+🤍💚❤️
+
+Това репо предоставя JSON описания на българските езикови модели [**BgGPT**](https://bggpt.ai),
+подходящи за инсталиране в чатбота [**Jan**](https://jan.ai).
+
+[**Jan**](https://jan.ai) е алтернатива на ChatGPT, която може да се използва
+локално и не изисква интернет връзка след сваляне на езиковите модели.
+
+
+📥 В [релийзите](../../releases) на това репо ще намерите .zip файлове, които
+съдържат JSON описания на моделите.
+
+
+## Как да използвам тези модели в Jan?
+1. Инсталирайте [Jan](https://jan.ai) на компютъра си, ако вече не сте го направили.
+2. От [последния релийз](../../releases/latest) на това репо изтеглете .zip файла.
+3. Разархивирайте съдържанието на .zip файла в директорията на Jan:
+    - Linux и macOS: `~/jan/models`
+    - Windows: `C:\Users\<your_user_name>\jan\models`
+4. Рестартирайте Jan, ако в момента е стартиран.
+5. От менюто **Hub** в Jan намерете и изтеглете избрания модел.
+
+## Кой модел да избера?
+BgGPT идва в няколко различни квантувани размера. Може да изберете модел, който
+да отговаря на вашите нужди, като вземете предвид размера на диска и
+изискването за VRAM.
+
+ℹ️ По-малките модели в повечето случаи са по-бързи при по-малко
+налична VRAM, но генерират по-некачествени резултати.
+
+
+| Модел | Размер върху диска | Изискване за VRAM |
+| --- | --- | --- |
+| `Q4_K_S` | 4.17 GB | 5.21 GB  |
+| `Q4_K_M` ✨ | 4.40 GB | 5.50 GB  |
+| `Q5_K_S` ✨ | 5.03 GB | 6.29 GB  |
+| `Q5_K_M` ✨ | 5.17 GB | 6.46 GB  |
+| `Q6_K`   | 5.98 GB | 7.48 GB  |
+| `Q8_0`   | 7.75 GB | 9.69 GB  |
+| `F16`    | 14.6 GB | 18.25 GB |
+
+
+> ✨ Препоръчани модели, базирани на [тази дискусия в llama.cpp](
+    https://github.com/ggerganov/llama.cpp/discussions/2094
+).
+> VRAM = 1.25 * Disk
+
+## Връзки
+- [INSAIT @ Hugging Face](https://huggingface.co/INSAIT-Institute)
+- [Jan @ GitHub](https://github.com/janhq/jan)

utils/model-generator/main.py

@@ -0,0 +1,47 @@
+import json
+import os
+
+import requests
+
+THIS_DIR = os.path.dirname(__file__)
+REPO_ROOT = os.path.join(THIS_DIR, "..", "..")
+MODELS_DIR = os.path.join(REPO_ROOT, "models")
+
+
+def main():
+    files = []
+
+    with open(os.path.join(THIS_DIR, "models_list.json"), "r") as fd:
+        models_list = json.load(fd)
+
+    for model_object in models_list:
+        repo = model_object["repo"]
+        gguf_files = model_object["gguf_files"]
+
+        for filename in gguf_files:
+            file_url = f"{repo}/resolve/main/{filename}"
+
+            head_resp = requests.head(file_url, allow_redirects=True)
+            files.append((file_url, filename, head_resp.headers["Content-Length"]))
+
+        with open(os.path.join(os.path.dirname(__file__), "model.template.json"), "r") as template:
+            template_content = template.read()
+
+            for file_url, filename, size in files:
+                filename_no_ext = os.path.splitext(filename)[0]
+                filled_template = template_content \
+                    .replace("<<FILENAME>>", filename) \
+                    .replace("<<FILENAME_NO_EXT>>", filename_no_ext) \
+                    .replace("<<FILE_URL>>", file_url) \
+                    .replace("<<SIZE>>", size)
+
+                os.makedirs(os.path.join(
+                    MODELS_DIR, filename_no_ext
+                ), exist_ok=True)
+
+                with open(os.path.join(MODELS_DIR, filename_no_ext, "model.json"), "w", newline="\n") as model:
+                    model.write(filled_template)
+
+
+if __name__ == "__main__":
+    main()

utils/model-generator/model.template.json

@@ -0,0 +1,36 @@
+{
+  "sources": [
+    {
+      "filename": "<<FILENAME>>",
+      "url": "<<FILE_URL>>"
+    }
+  ],
+  "id": "<<FILENAME_NO_EXT>>",
+  "object": "model",
+  "name": "<<FILENAME_NO_EXT>>",
+  "version": "0.1",
+  "description": "Bulgarian language model",
+  "format": "gguf",
+  "settings": {
+    "ctx_len": 4096,
+    "prompt_template": "[INST] {prompt} [/INST]",
+    "llama_model_path": "<<FILENAME>>"
+  },
+  "parameters": {
+    "temperature": 0.7,
+    "top_p": 0.95,
+    "stream": true,
+    "max_tokens": 4096,
+    "stop": [],
+    "frequency_penalty": 0,
+    "presence_penalty": 0
+  },
+  "metadata": {
+    "author": "BgGPT",
+    "tags": [
+      "7B"
+    ],
+    "size": <<SIZE>>
+  },
+  "engine": "nitro"
+}

utils/model-generator/models_list.json

@@ -0,0 +1,26 @@
+[
+  {
+    "repo": "https://huggingface.co/INSAIT-Institute/BgGPT-7B-Instruct-v0.1-GGUF",
+    "gguf_files": [
+      "BgGPT-7B-Instruct-v0.1.F16.gguf",
+      "BgGPT-7B-Instruct-v0.1.Q4_K_M.gguf",
+      "BgGPT-7B-Instruct-v0.1.Q4_K_S.gguf",
+      "BgGPT-7B-Instruct-v0.1.Q5_K_M.gguf",
+      "BgGPT-7B-Instruct-v0.1.Q5_K_S.gguf",
+      "BgGPT-7B-Instruct-v0.1.Q6_K.gguf",
+      "BgGPT-7B-Instruct-v0.1.Q8_0.gguf"
+    ]
+  },
+  {
+    "repo": "https://huggingface.co/INSAIT-Institute/BgGPT-7B-Instruct-v0.2-GGUF",
+    "gguf_files": [
+      "BgGPT-7B-Instruct-v0.2.F16.gguf",
+      "BgGPT-7B-Instruct-v0.2.Q4_K_M.gguf",
+      "BgGPT-7B-Instruct-v0.2.Q4_K_S.gguf",
+      "BgGPT-7B-Instruct-v0.2.Q5_K_M.gguf",
+      "BgGPT-7B-Instruct-v0.2.Q5_K_S.gguf",
+      "BgGPT-7B-Instruct-v0.2.Q6_K.gguf",
+      "BgGPT-7B-Instruct-v0.2.Q8_0.gguf"
+    ]
+  }
+]

utils/model-generator/requirements.txt

@@ -0,0 +1 @@
+requests==2.31.0