Integration with open source, local LLMs

[jeffpaul]

Is your enhancement related to a problem? Please describe.

There are varying options on open source, local LLMs that we could look to integrate with ClassifAI. These come on the market with some regularity, so we might want to consider not just direct integration with individual models but perhaps a middleware layer that would then integrate with new OSS models as the become available. This would allow ClassifAI users to have a free, open source option for a service provider that would also allow for controlling where their site data goes. So a data privacy and cost perspective win.

Designs

No response

Describe alternatives you've considered

No immediate options that I'd recommend we start with, so some exploration on options would be best before proceeding with any integrations.

Code of Conduct

• I agree to follow this project's Code of Conduct