Add context for LLM

[btsogt21]

How can we go about setting up 'context' for the LLM to respond based on? For example, I would like to feed it information about myself and get responses tailored to said information.

Along a similar line, how could we go about implementing conversation 'memory' for the LLM when requesting responses?

[abhinavuppal1]

How can we go about setting up 'context' for the LLM to respond based on? For example, I would like to feed it information about myself and get responses tailored to said information.

Please elaborate a little further with a use case.
The more information you are able to provide us, the better answer I shall be able to provide.

Along a similar line, how could we go about implementing conversation 'memory' for the LLM when requesting responses?

For getting responses from LLM, part of the conversation is sent to the API. The number of earlier conversation segments sent to the API are governed by constant MAX_TRANSCRIPTION_PHRASES_FOR_LLM (default 12) defined here

[raulvasquez]

i believe by context he means providing the model your background. For example using this on a zoom interview being able to feed it your resume, or the job description.

[tan-banana]

I put the llm directions in the you: first statement . and the resume context in the other file. However when i downlaod the new pull request or whatever it results in bugs. there must be a way to control better the thing that is sent to the llm to respond. transcription has been good for me on the free version, and the cost to run the rest is very low cost.

[raulvasquez]

@vivekuppal I am unsure if this will work. I notice the parameters.yaml says the # Application expects role "You" to have 1 entry, but can there be more than 1 entry in the initial convo block to provide more context and background to the assistant?

General:
  system_prompt: |
    You are assisting an experienced manager interviewing for a position at GLOBAL CORP as the OPERATIONS MANAGER. Respond in detail to the conversation. Confidently give a straightforward response to the speaker, even if you don't understand them. Give your response in square brackets. DO NOT ask to repeat, and DO NOT ask for clarification. Just answer the speaker directly.
  initial_convo:
    first:
      role: "You"
      content: |
        Hey assistant, how are you doing today, I am Manager. I will be interviewing for a job as the OPERATIONS MANAGER at GLOBAL CORP. I will share my resume first and then the job description.

    second:
      role: "assistant"
      content: "Hello, Manager. You are awesome. I am doing very well and looking forward to the OPERATIONS MANAGER discussion with you. Please send your resume first."
    
    third:
      role: "You"
      content: |      
        Here is my resume:
        ---
        ### NAME
        ---
        HEADING
        ---
        ### WORK EXPERIENCE
        USA CORP
        MANAGER
        2006 - 2012
        NEW YORK, NY
        * Experience Bullet 1
        * Experience Bullet 2
        * Experience Bullet 3
        ---
        ### INDIVIDUAL PROJECTS
        PROJECT NAME
        07/2021 - Present
        * Project Bullet 1
        * Project Bullet 2
        ---
        ### SKILLS
        Management Skills: Skill 1, Skill 2, Skill 3

        Technical Skills: Excel, Python, GPT
        ---
        ### EDUCATION
        University of USA, 2008-2011
        * Major: Management
        * Minor: CS
        * GPA: 4.0
        ---

    fourth:
      role: "assistant"
      content: "Thank you for sharing your resume. Please send the job description."
    
    fifth:
      role: "You"
      content: |
        Here is the job description:

        OPERATIONS MANAGER @ GLOBAL CORP

        JOB DESCRIPTION:
        As an OPERATIONS MANGER you will:
        * Manger Operations
        * Get Paid

        Basic Qualifications: 
        * Bachelors Degree
        * Work experience

        About GLOBAL CORP:
        We are a global technology company focused on OPERATIONS and stuff.

    sixth:
      role: "assistant"
      content: "Thank you for sharing the job description and resume. I am happy to assist with your interview."

[tan-banana]

• Add toggle using preset contexts changable in the UI. Like I want to presave multiple contexts for different personas. And be able to switch them live time.

[chenosyong]

• Add a switch using a default context that can be changed in the UI. Just like I want to pre-save multiple contexts for different characters. And being able to switch them in real time.

Hello friend, can you share the code

[vivekuppal]

Hi Everyone, @tan-banana, @btsogt21, @chenosyong, @raulvasquez,
It seems like we are attempting to solve a few problems for some specific use cases in transcribe. I would be happy to help move your needs forward. The best way to collaborate would be to share the source code. As Transcribe code base moves forward, it might become difficult for you to keep your personal code base copy up to date with new changes.

I am eager to help move things forward.

[vivekuppal]

看起来我们正在尝试解决转录中某些特定用例的一些问题。 我很乐意帮助您满足您的需求。 最好的协作方式是共享源代码。 随着 Transcribe 代码库的不断发展，您可能很难使您的个人代码库副本与新更改保持同步。

我渴望帮助推动事情向前发展。

[deesinha]

I would be happy to see this use case included:

User Story: Personalized Responses Based on User Background
As a user, I want to provide relevant background information to the LLM (Language Model) so that it can tailor its responses to my specific context. This will enhance the quality and relevance of the LLM’s interactions with me.
Acceptance Criteria:
User Input: The LLM should allow users to input details about their background, including job experience, skills, and project history.
Contextual Adaptation: When responding to user queries, the LLM should consider the provided background information and adjust its answers accordingly.
Personalization: The LLM’s responses should reflect an understanding of the user’s unique context, preferences, and expertise.
Privacy and Security: The LLM must handle user-provided information securely and ensure that no sensitive data is exposed.
Feedback Loop: Users should have the ability to update or modify their background information over time, allowing the LLM to adapt continuously.