Add local-ai community container with Vulkan support

[docjyJ]

A lot of GPU supported: https://vulkan.gpuinfo.org/listdevices.php?platform=linux

Image: https://github.com/docjyJ/aio-local-ai-vulkan

[szaimen]

@docjyJ thanks a lot for this PR! :)

[szaimen]

Btw @docjyJ have you had the chance to look a bit further into this? :)

[docjyJ]

Not yet, it's quite complicated to have Vulkan on my server. I have a very old version of the Linux kernel so I need to find a solution to update.

[docjyJ]

Local AI doesn't have a Vulkan image for arm64...
Since my cloud is on arm, I'm stuck...

I've opened an issue: mudler/LocalAI#5778

[docjyJ]

I have Vulkan on my PC and am running a local instance of Nextcloud over HTTP, so I opted for simplicity with automatic configuration and explanations for accessing the web interface.

[szaimen]

Thanks @docjyJ for continuing the work on this! 😊

I mean you could also try to build the image from source instead of proxying it. Then you could also build for arm64... However of course not sure how feasible this is...

[docjyJ]

It's using my GPU. But I don't know why it's so slow...

[docjyJ]

So WDYT ?

[docjyJ]

See that https://www.reddit.com/r/LocalLLaMA/comments/1j1swtj/vulkan_is_getting_really_close_now_lets_ditch/

[szaimen]

So WDYT ?

I was wondering if it would be possible to make this container the default LocalAI container in AIO instead of being a variant?

Of course, we would need to resolve the arm64 problem first.

Would it be possible to simply build the project from sources in the Dockerfile? WDYT?

[docjyJ]

Yes, I think it could be the main container.

ARM support has an open PR: mudler/LocalAI#5780

[szaimen]

Yes, I think it could be the main container.

Cool!

ARM support has an open PR: mudler/LocalAI#5780

I see... Honestly, I would like to wait a few weeks and see if arm64 support evolves upstream so that we can push this forward here...

[docjyJ]

yes !

[docjyJ]

I added web ui access with basic auth, we can support caddy community container.
Updated doc and use local AI image without preconfigured models (to save space disk, user can install models from local AI web ui)

[szaimen]

FYI: you can now use string replacement in nextcloud_exec_commands since #6835. See #6835 for example.

[docjyJ]

I saw that PR has had the Roadmap label for two weeks. Waiting for news.