When to open the knowledge base automatically

[ruKurz]

Spend some hours in order to make a suggestion when to open the knowledge base.
Think about a metric ...

[westei]

IMO the general rule is to show the knowledge base if a QueryProvider is highly relevant to the current state of the conversation. Further relevancy is also a function of specialisation of the query provider. A very generic one (e.g. Information Retrieval) has some relevancy for every state of the conversation). Very specific (e.g. Reiseplanung) is only valid in special cases but than it can be considered as highly relevant.

So in general the challenge is

• for generic QueryProvider to find a metric for the relevancy of the results
• for specific QueryProvider its the precision and recall of their activation

One issue with the current setup in Smarti is that we do only provide rather generic QueryProviders that all do some kind of information retrieval. This means that the only area of improvement lies in the relevancy of the results.

Based on this the goal for the Expertengespräche MUST BE to present results that are good enough so that it can always shown to the user. IMHO the combination of related conversation and similarity based conversation search in 0.7.1 goes a long way into this direction. Work needs still do be done in the presentation of results. Especially the expansion of the context has a lot of room for improvement. Also the representation of single Messages.

For external Search-Providers the metric is the real challenge. To establish a such we do need privileged access to the Search-Provider. Possibilities include (a) use TF-IDF for extracted Tokens (b) access to Solr MLT to retrieve Interesting Terms (c) perform the query and analyse metadata about the results. (a) and (b) are not possible without privileged access to the Search-Provider and (c) allows only for rather weak metrics (e.g. number of results, gabs in the scores of results ...). Note also that everything needs to be implemented in the Widget as otherwise we would need to add the requirement that the Smarti Backend needs to have access to the Search-Provider.

To further improve relevancy we would need to investigate low hanging fruits for specific QueryProvider. In the travel assistent setting we used to have specific query providers for travel planing and radial search. Those allowed to provide very targeted information to users. Also the UI of the widget allows for very specific interactions and to perform relevant actions. In the context of Smarti we do not have this type of query providers at the moment so we are missing out on such kind of opportunities.

[ruKurz]

Thanks for that detailed analysis.