Improvement: Chat Feature Tracker

[rmusser01]

Issue is to track improvements to the chat Functionality

• Hook up Top/Min P values to config.txt values.
• Streaming response handling
• chat history is summarized with each new message and only the summary + the last N messages are sent to the model instead of the whole chat
• Chat branching/forking + swapping branches
• Prompt template presets
  - Jinja templates? Enhancement: Fully Implement Chat Dictionaries/Flesh out implementation #480
• Sampling parameter presets (Min-P, Top-P, Top-K and temperature are in)
• Lorebook management
• Prioritization of inclusion of messages/DBs for context - level 1, level 2, level 3
• Depth anchors for lorebooks
• Real-time voice chat with response/audio detection
• Option for command callouts via chatbox - !webrag www.example.com/blogpost1 What is RAG? - command identified as user wanting website ingested into RAG DB, and then perform a search across it using the query What is RAG? / etc/same idea
• Custom dictionary support for Jinja templates/replacing
• Display total chat history token count + current chat window token input count + combined total of both
• [ ]

[rmusser01]

https://abbey.us.ai/

SillyTavern/SillyTavern#2189

At present, we can choose the specific depth of lorebook entries, but we cannot rapidly change the depth of most lorebook entries. This becomes a problem when switching models, because some models are good at reading information at higher depths, while others require information at much lower depths.

For example, lets say that I have a page full of lorebook entries at various depths. I want to subtract 2 from the depth of all of my lorebook entries as I switch models. For some of those lorebook entries, I could just tether them to the author's note and change the depth of the author's note. However, my lorebook entries span a wide variety of depths. Important information is set to a lower depth, while more general information is set to a higher depth.

So, if I tether the author's note to 'critically important information', then I'll still need to manually change the depth of all lorebook entries that are not 'critically important information'. This requires going through each entry one by one, and can be a very time consuming process.
Describe the solution you'd like

I propose the creation of new depth categories. That way, instead of setting the depth of each individual entry, I could instead tether the depth of each lorebook entry to a global category, similar to how many use the author's note category.

A few category suggestions:

    Directive information
    Priority information
    Important Information
    Scenario Information
    General Information

Using these examples, I could set directive information to (D1), priority information to (D2), important information to (D5), scenario information to (D7), and general information to (D10).

Using this system I could rapidly change the depth of all relevant entries by changing the depth of a single global variable. Ex, if a model doesn't seem to be reading my 'general information' very well, then I could set the depth to (D8) instead of (D10). If my priority information seems to be stifling my model, I could change it to (D3) instead of (D2).

I am planning to tackle this feature soon. You will be able to choose "Depth Anchors" that you can define as many as you want and name how you want. Each anchor will have a name, an attached role and depth setting. And the WI can connect to an anchor.
If you have a few dozen WI connected to an anchor for example called "Directive Information" at depth 5 and then change that to depth 3, next generation all WI with that anchor will be inserted together at depth 3 instead of 5.

[rmusser01]

https://arxiv.org/abs/2411.07641