Add WebSocket streaming for real-time responses

[kartikeyg0104]

Add WebSocket streaming for real-time responses

Description

This pull request introduces WebSocket streaming functionality to the AI Chatbot plugin. The new feature enables token-by-token streaming of responses, improving responsiveness and user experience. This is particularly beneficial for local LLM setups, as it allows real-time output rendering in the frontend.

Changes Made

1. Backend Updates:

   • Added WebSocket endpoint /sessions/{session_id}/stream in chatbot-core/api/routes/chatbot.py.
   • Implemented token-by-token streaming in chatbot-core/api/models/llama_cpp_provider.py and chatbot-core/api/services/chat_service.py.

2. Frontend Demo:

   • Created websocket_demo.html to showcase real-time streaming in a simple chat interface.

3. Test Scripts:

   • Added test scripts (test_websocket_simple.py, test_streaming_direct.py) to verify WebSocket functionality.

4. Updated .gitignore:

   • Excluded test scripts, logs, and demo files from being committed to the repository.

Testing Done

• Verified WebSocket connection and token streaming using test scripts.
• Tested session creation and token-by-token response generation.
• Confirmed backward compatibility with existing REST API endpoints.

Benefits

• Improved UX: Real-time responses enhance user experience.
• Backward Compatibility: REST API remains unchanged.
• Extensibility: WebSocket streaming can be integrated into the Jenkins plugin frontend.

[berviantoleo]

Hi @kartikeyg0104,

Thank you for contributing.

I have some general feedback:

1. Please use a feature branch in your fork repo when creating a pull request. So, whenever we have any updates in this repo (especially the main branch), you still have a similar history/commits.
2. I only see changes to api code. Did you forget to include the frontend and test codes?

[kartikeyg0104]

Thank you so much for your feedback, Sir! I really appreciate the suggestion about using a feature branch, and I’ll make sure to follow this practice in future pull requests to maintain a clean history and avoid conflicts with updates in the main branch. As for the frontend, I missed connecting the WebSocket functionality to the frontend. I’ll work on integrating and testing the WebSocket connection and include the necessary changes in a follow-up commit.

[berviantoleo]

As I mentioned before, it looks like your PR descriptions and the actual changes are not matched.

Please at least, add the testing code or the front end code. Any codes that's not related to the purpose of the PR shouldn't be mentioned here.

[berviantoleo]

Don't need to add these files into .gitignores.

[berviantoleo]

We shouldn't send the error to the client. We may log it, but please send a general error text.