Code example to build Gen AI POC application using Gradio and Gen AI …

…agents. (#1092) * test access * genai and gradio code sample * genai and gradio code sample * adding link to readme in the root folder. * removing the unwanted line.
GoogleCloudPlatform · Sep 13, 2023 · cf1cf9d · cf1cf9d
1 parent e0e7dd8
commit cf1cf9d
Show file tree

Hide file tree

Showing 19 changed files with 880 additions and 0 deletions.
diff --git a/README.md b/README.md
@@ -554,6 +554,8 @@ Platform usage.
 *   [Composer DAG Load Generator](tools/cloud-composer-dag-generator) - This is an automatic 
     DAG generator tool which can be used to create test workload on a cloud composer environmnet
     and to test differents airflows configurations or to do fine tune using the composer/airflow metrics.
+*   [Gradio and Generative AI Example](examples/genai-gradio-example) - The example code allows developers 
+    to create rapid Generative AI PoC applications with Gradio and Gen AI agents.
 
 ## Contributing
 

diff --git a/examples/genai-gradio-example/README.md b/examples/genai-gradio-example/README.md
@@ -0,0 +1,104 @@
+```
+Copyright 2023 Google. This software is provided as-is, without warranty or
+representation for any use or purpose. Your use of it is subject to your
+agreement with Google.
+```
+## Technology Stack
+- Google Cloud Run
+- Google Artifact Registry
+- Google Cloud Storage
+- Google Speech to Text
+- Vertex AI Conversation
+- Dialogflow CX
+- Dialogflow CX Agent
+- Google Data Store
+- Google Secret Manager
+- Gradio
+
+## GCP Project Setup
+
+### Creating a Project in the Google Cloud Platform Console
+
+If you haven't already created a project, create one now. Projects enable you to
+manage all Google Cloud Platform resources for your app, including deployment,
+access control, billing, and services.
+
+1. Open the [Cloud Platform Console][cloud-console].
+2. In the drop-down menu at the top, select **NEW PROJECT**.
+3. Give your project a name.
+4. Make a note of the project ID, which might be different from the project
+   name. The project ID is used in commands and in configurations.
+
+[cloud-console]: https://console.cloud.google.com/
+
+### Enabling billing for your project.
+
+If you haven't already enabled billing for your project, [enable
+billing][enable-billing] now. Enabling billing allows is required to use Cloud
+Bigtable and to create VM instances.
+
+[enable-billing]: https://console.cloud.google.com/project/_/settings
+
+### Install the Google Cloud SDK.
+
+If you haven't already installed the Google Cloud SDK, [install the Google
+Cloud SDK][cloud-sdk] now. The SDK contains tools and libraries that enable you
+to create and manage resources on Google Cloud Platform.
+
+[cloud-sdk]: https://cloud.google.com/sdk/
+
+### Setting Google Application Default Credentials
+
+Set your [Google Application Default
+Credentials][application-default-credentials] by [initializing the Google Cloud
+SDK][cloud-sdk-init] with the command:
+
+```
+   gcloud init
+```
+
+Generate a credentials file by running the
+[application-default login](https://cloud.google.com/sdk/gcloud/reference/auth/application-default/login)
+command:
+
+```
+    gcloud auth application-default login
+```
+
+[cloud-sdk-init]: https://cloud.google.com/sdk/docs/initializing
+
+[application-default-credentials]: https://developers.google.com/identity/protocols/application-default-credentials
+
+## Upload your data to a Cloud Storage bucket
+Follow these [instructions][instructions] to upload your pdf documents 
+or pdf manuals to be used in this example
+
+[instructions]:https://cloud.google.com/storage/docs/uploading-objects
+
+## Create a Generative AI Agent
+Follow the instructions at this [link][link] and perform the following:
+1. Create Data Stores: Select information that you would like the Vertex AI Search and Conversation to query
+2. Create an Agent: Create the Dialogflow CX agent that queries the Data Store
+3. Test the agent in the simulator
+4. Take note of you agent link by going to [Dialogflow CX Console][Dialogflow CX Console] and see the information about the agent you created
+
+[link]: https://cloud.google.com/generative-ai-app-builder/docs/a
+[Dialogflow CX Console]:https://cloud.google.com/dialogflow/cx/docs/concept/console#agent
+
+### Dialogflow CX Agent Data Stores
+Data Stores are used to find answers for end-user's questions. 
+Data Stores are a collection documents, each of which reference your data.
+
+For this particular example data store will consist of the following characteristics:
+1. Your organizational documents or manuals.  
+2. The data store type will be unstructured in a pdf format
+3. The data is uploaded without metadata for simplicity.
+Only need to point the import to the gcp bucket folder where the pdf files are. 
+Their extension will decide their type.
+
+When an end-user asks the agent a question, the agent searches for an answer from the 
+given source content and summarizes the findings into a coherent agent response. 
+It also provides supporting links to the sources of the response for the end-user to learn more. 
+
+
+
diff --git a/examples/genai-gradio-example/frontend/Dockerfile b/examples/genai-gradio-example/frontend/Dockerfile
@@ -0,0 +1,10 @@
+FROM python:3.10 
+WORKDIR /usr/src/app 
+
+RUN apt-get update && apt-get install -y 
+RUN apt-get install ffmpeg -y
+COPY requirements.txt ./ 
+RUN pip install --no-cache-dir -r requirements.txt 
+COPY . . 
+EXPOSE 8080
+CMD [ "python", "./app/main.py" ] 
diff --git a/examples/genai-gradio-example/frontend/README.md b/examples/genai-gradio-example/frontend/README.md
@@ -0,0 +1,64 @@
+```
+Copyright 2023 Google. This software is provided as-is, without warranty or
+representation for any use or purpose. Your use of it is subject to your
+agreement with Google.
+```
+
+# Running voice-activated chatBot on Linux
+
+# Enable APIs on GCP
+The following is a list of the APIs that need to be enabled in GCP
+- Speech-to-text API.
+- Secret Manager API
+
+## Install Libraries
+```
+$ sudo apt-get install python3-pip python-dev
+$ sudo apt-get install ffmpeg
+```
+
+## Install Dependencies
+```
+$ pip install gradio==3.38.0 --use-deprecated=legacy-resolver
+$ pip install --upgrade google-cloud-speech==2.21.0
+$ pip install torch
+```
+
+If you face space issue try increasing CPU and Memory or you can create temp folder to cache torch in different folder as shown below:
+```
+$ pip install --cache-dir=/home/user/tmp torch
+```
+
+```
+$ pip3 install google-cloud-secret-manager==2.10.0
+$ pip3 install google-cloud-speech==2.21.0
+```
+
+## Service Account Access
+The service account you're using to run the application shpuld have below IAM roles:
+- Secret Manager Aecret Accessor
+- Cloud Run Invoker (required to call LLM middleware if middleware is deployed in Cloud Run)
+
+## Replace following parameters in ./app/config.ini to your value
+```
+SECRET_ID_IN_SECRET_MANAGER_FOR_PASSWORD: this is the secret id for the password secret created in the secret manager.
+SECRET_MANAGER_PROJECT_ID: gcp project id where the application will load secrets from.
+LLM_MIDDLEWARE_HOST_URL: host url for LLM middleware. URL looks like https://llm-middleware-sdnkdn12.a.run.app if the application is deployed in Cloud Run.
+```
+
+## How to run the application 
+From the frontend/app folder run the following command
+```
+$ python3 main.py
+```
+## How to enable SSL and expose the application outside of VM
+To enable SSL encryption and expose the application port outside of the VM, use the following launch command 
+in main.py:
+
+```
+$ bot_interface.launch(server_name="0.0.0.0", 
+      share=False,
+      ssl_certfile="localhost.crt", 
+      ssl_keyfile="localhost.key", 
+      ssl_verify=False)
+```
diff --git a/examples/genai-gradio-example/frontend/app/bot_interface.py b/examples/genai-gradio-example/frontend/app/bot_interface.py
@@ -0,0 +1,130 @@
+# ============================================================================
+# Copyright 2023 Google. This software is provided as-is, without warranty or
+# representation for any use or purpose. Your use of it is subject to your
+# agreement with Google.
+# ============================================================================
+
+"""Generate components to render user interface."""
+
+from event_handlers import EventHandlers as handlers
+import gradio as gr
+
+
+class BotInterface:
+  """Defines user interface."""
+
+  def initialize(self, config) -> any:
+    """Initialize and returns the chatbot interface with event handlers attached.
+
+    Args:
+            config: A configparser having gradio configurations loaded from
+              config.ini file.
+
+    Returns:
+            gr (Gradio): Gradio block consisting UI components.
+    """
+    event_handlers = handlers(config)
+    with gr.Blocks(
+        css=".gradio-container {border: 1px solid #e5e5e5}"
+    ) as bot_interface:
+      session = gr.State([])
+      with gr.Row():
+        with gr.Column(scale=10):
+          with gr.Row(scale=1):
+            chatbot = gr.Chatbot(
+                [(None, config["initial-message"])],
+                elem_id="chatbot",
+                show_label=False,
+                height=640,
+            )
+          with gr.Row():
+            with gr.Column(scale=12):
+              user_input = gr.Textbox(
+                  show_label=False,
+                  placeholder=config["text-placeholder"],
+                  container=False,
+              )
+            with gr.Column(min_width=70, scale=1):
+              submit_btn = gr.Button("Send")
+          with gr.Row():
+            audio_input = gr.Audio(source="microphone", type="filepath")
+          with gr.Row():
+            clear_btn = gr.Button("Start a new conversation")
+          with gr.Row():
+            with gr.Column(scale=1):
+              source_location = gr.Textbox(
+                  label=config["location-label"],
+                  show_label=True,
+                  interactive=False,
+                  show_copy_button=True,
+                  lines=10,
+                  max_lines=10,
+              )
+      input_msg = user_input.submit(
+          event_handlers.add_user_input,
+          [chatbot, user_input],
+          [chatbot, user_input],
+          queue=False,
+      ).then(
+          event_handlers.bot_response,
+          [chatbot, session],
+          [
+              chatbot,
+              session,
+              source_location,
+          ],
+      )
+      submit_done = submit_btn.click(
+          event_handlers.add_user_input,
+          [chatbot, user_input],
+          [chatbot, user_input],
+          queue=False,
+      ).then(
+          event_handlers.bot_response,
+          [chatbot, session],
+          [
+              chatbot,
+              session,
+              source_location,
+          ],
+      )
+      clear_btn.click(
+          event_handlers.clear_history,
+          [chatbot, session],
+          [
+              chatbot,
+              session,
+              source_location,
+          ],
+          queue=False,
+      )
+      input_msg.then(
+          lambda: gr.update(interactive=True), None, [user_input], queue=False
+      )
+      submit_done.then(
+          lambda: gr.update(interactive=True), None, [user_input], queue=False
+      )
+      inputs_event = (
+          audio_input.stop_recording(
+              event_handlers.transcribe_file, audio_input, user_input
+          )
+          .then(
+              event_handlers.add_user_input,
+              [chatbot, user_input],
+              [chatbot, user_input],
+              queue=False,
+          )
+          .then(
+              event_handlers.bot_response,
+              [chatbot, session],
+              [
+                  chatbot,
+                  session,
+                  source_location,
+              ],
+          )
+      )
+      inputs_event.then(
+          lambda: gr.update(interactive=True), None, [user_input], queue=False
+      )
+    return bot_interface
diff --git a/examples/genai-gradio-example/frontend/app/config.ini b/examples/genai-gradio-example/frontend/app/config.ini
@@ -0,0 +1,21 @@
+# ============================================================================
+# Copyright 2023 Google. This software is provided as-is, without warranty or
+# representation for any use or purpose. Your use of it is subject to your
+# agreement with Google.
+# ============================================================================
+
+[gradio]
+title=Gradio powered by Gen AI
+initial-message=Hello, how may I help you?
+text-placeholder=Enter Text
+location-label=Source Location
+error-response=Apologies, something went wrong. Please start a new conversation or try after sometime.
+
+[secret]
+gradio-user=admin
+password-secret-id=fea-admin-pass
+sm-project-id=ford-upshift-fea
+
+[api-config]
+llm-mw-endpoint=https://fea-llm-middleware-2z5uoxo2wa-uc.a.run.app
+get-llm-response=/predict