Question: Is it possible for Garak to scan LLMs served on vLLM or similar?

[vecorro]

Is there a way to point Garak to an HTTP to scan LLMs served via a REST API? or is it possible to provide a URL for an OpenAI compatible API service?

Thanks!

[leondz]

Not yet, but it should be. Will put this in the queue. Progress will be tracked in issue #286

[Eric-Hacker]

I noticed Friday that #286 has been closed, but the REST generator has not added to the docs. I haven't had time to dig through the code and other docs to try to piece together what is expected here. Any help is appreciated.

[leondz]

Indeed! The REST package code is now documented incl. example. Unfortunately readthedocs no longer accepts our config, so there's nothing on docs.garak.ai, and given RTD's user friendliness this might remain the case for a little while. But I encourage you to check out rest.py and send feedback!

[Eric-Hacker]

Docs look great in the code. Thank you! I should be able to start testing this week and will let you know how it goes.