|
2 | 2 | > [!WARNING] |
3 | 3 | > ARM devices are not supported by this container. It is only for x86_64 devices. (See: https://github.com/mudler/LocalAI/issues/5778) |
4 | 4 |
|
5 | | -> [!CAUTION] |
6 | | -> If the configuration is not correct, it should still work but it will use the CPU only and will not benefit from hardware acceleration. |
7 | | -
|
8 | | -## Local AI |
9 | | -This container bundles Local AI and auto-configures it for you. |
| 5 | +## Local AI with Vulkan support |
| 6 | +This container bundles Local AI and auto-configures it for you. It support hardware acceleration with Vulkan. |
10 | 7 |
|
11 | 8 | ### Notes |
12 | | -- Make sure vulkan is enabled in the AIO settings. Run `vulkaninfo` in the terminal to check if it is enabled. |
13 | | -- Make sure to have enough storage space available. This container alone needs ~7GB storage. |
14 | | -- Make sure to have enabled DRI device by adding `--env NEXTCLOUD_ENABLE_DRI_DEVICE=true`, see https://github.com/nextcloud/all-in-one?tab=readme-ov-file#with-open-source-drivers-mesa-for-amd-intel-and-new-drivers-nouveau-for-nvidia. |
15 | | -- To make it work, you first need to browse `https://your-nc-domain.com/settings/admin/ai` and enable or disable specific features for your models in the openAI settings. Afterwards using the Nextcloud Assistant should work. |
16 | | -- To access the Local AI web interface, you need to set reverse proxy rules for it. |
17 | | -```Cadyfile |
18 | | -http://local-ai.your-nc-domain.com { |
19 | | - # Local AI web interface haven't any authentication, so you should protect it |
20 | | - basic_auth { |
21 | | - # Username "Bob", password "hiccup" |
22 | | - Bob $2a$14$Zkx19XLiW6VYouLHR5NmfOFU0z2GTNmpkT/5qqR7hx4IjWJPDhjvG |
23 | | - } |
24 | | - reverse_proxy nenxtcloud-aio-local-ai-vulkan:8080 |
25 | | -} |
26 | | -``` |
| 9 | +Documentation is available on the container repository. This documentation is regularly updated and is intended to be as simple and detailed as possible. Thanks for all your feedback! |
| 10 | + |
| 11 | +- See https://github.com/docjyJ/aio-local-ai-vulkan#getting-started for getting start with this container. |
27 | 12 | - See [this guide](https://github.com/nextcloud/all-in-one/discussions/5430) for how to improve AI task pickup speed |
28 | 13 | - See https://github.com/nextcloud/all-in-one/tree/main/community-containers#community-containers how to add it to the AIO stack |
29 | 14 | - Note that Nextcloud supports only one server for AI queries, so this container cannot be used at the same time as other AI containers. |
|
0 commit comments