Merge pull request #472 from harrsshh08/main

Community contribution: 1_foundations using gemini

Author: ed-donner

1_foundations/community_contributions/1_foundations_using_gemini/1_lab1.ipynb

@@ -0,0 +1,382 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Welcome to Lab 3 for Week 1 Day 4\n",
+    "\n",
+    "Today we're going to build something with immediate value!\n",
+    "\n",
+    "In the folder `me` I've put a single file `linkedin.pdf` - it's a PDF download of my LinkedIn profile.\n",
+    "\n",
+    "Please replace it with yours!\n",
+    "\n",
+    "I've also made a file called `summary.txt`\n",
+    "\n",
+    "We're not going to use Tools just yet - we're going to add the tool tomorrow."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<table style=\"margin: 0; text-align: left; width:100%\">\n",
+    "    <tr>\n",
+    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
+    "            <img src=\"../assets/tools.png\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
+    "        </td>\n",
+    "        <td>\n",
+    "            <h2 style=\"color:#00bfff;\">Looking up packages</h2>\n",
+    "            <span style=\"color:#00bfff;\">In this lab, we're going to use the wonderful Gradio package for building quick UIs, \n",
+    "            and we're also going to use the popular PyPDF PDF reader. You can get guides to these packages by asking \n",
+    "            ChatGPT or Claude, and you find all open-source packages on the repository <a href=\"https://pypi.org\">https://pypi.org</a>.\n",
+    "            </span>\n",
+    "        </td>\n",
+    "    </tr>\n",
+    "</table>"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# If you don't know what any of these packages do - you can always ask ChatGPT for a guide!\n",
+    "\n",
+    "from dotenv import load_dotenv\n",
+    "from openai import OpenAI\n",
+    "from pypdf import PdfReader\n",
+    "import os\n",
+    "import gradio as gr"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "load_dotenv(override=True)\n",
+    "GEMINI_BASE_URL = \"https://generativelanguage.googleapis.com/v1beta/openai/\"\n",
+    "google_api_key = os.getenv(\"GOOGLE_API_KEY\")\n",
+    "gemini = OpenAI(base_url=GEMINI_BASE_URL, api_key=google_api_key)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "reader = PdfReader(\"me/linkedin.pdf\")\n",
+    "linkedin = \"\"\n",
+    "for page in reader.pages:\n",
+    "    text = page.extract_text()\n",
+    "    if text:\n",
+    "        linkedin += text"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "print(linkedin)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "with open(\"me/summary.txt\", \"r\", encoding=\"utf-8\") as f:\n",
+    "    summary = f.read()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "name = \"Harsh Patidar\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "system_prompt = f\"You are acting as {name}. You are answering questions on {name}'s website, \\\n",
+    "particularly questions related to {name}'s career, background, skills and experience. \\\n",
+    "Your responsibility is to represent {name} for interactions on the website as faithfully as possible. \\\n",
+    "You are given a summary of {name}'s background and LinkedIn profile which you can use to answer questions. \\\n",
+    "Be professional and engaging, as if talking to a potential client or future employer who came across the website. \\\n",
+    "If you don't know the answer, say so.\"\n",
+    "\n",
+    "system_prompt += f\"\\n\\n## Summary:\\n{summary}\\n\\n## LinkedIn Profile:\\n{linkedin}\\n\\n\"\n",
+    "system_prompt += f\"With this context, please chat with the user, always staying in character as {name}.\"\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "system_prompt"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model_name = \"gemini-2.5-flash-preview-05-20\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def chat(message, history):\n",
+    "    messages = [{\"role\": \"system\", \"content\": system_prompt}] + history + [{\"role\": \"user\", \"content\": message}]\n",
+    "    response = gemini.chat.completions.create(model=model_name, messages=messages)\n",
+    "    return response.choices[0].message.content"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Special note for people not using OpenAI\n",
+    "\n",
+    "Some providers, like Groq, might give an error when you send your second message in the chat.\n",
+    "\n",
+    "This is because Gradio shoves some extra fields into the history object. OpenAI doesn't mind; but some other models complain.\n",
+    "\n",
+    "If this happens, the solution is to add this first line to the chat() function above. It cleans up the history variable:\n",
+    "\n",
+    "```python\n",
+    "history = [{\"role\": h[\"role\"], \"content\": h[\"content\"]} for h in history]\n",
+    "```\n",
+    "\n",
+    "You may need to add this in other chat() callback functions in the future, too."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "gr.ChatInterface(chat, type=\"messages\").launch()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## A lot is about to happen...\n",
+    "\n",
+    "1. Be able to ask an LLM to evaluate an answer\n",
+    "2. Be able to rerun if the answer fails evaluation\n",
+    "3. Put this together into 1 workflow\n",
+    "\n",
+    "All without any Agentic framework!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Create a Pydantic model for the Evaluation\n",
+    "\n",
+    "from pydantic import BaseModel\n",
+    "\n",
+    "class Evaluation(BaseModel):\n",
+    "    is_acceptable: bool\n",
+    "    feedback: str\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "evaluator_system_prompt = f\"You are an evaluator that decides whether a response to a question is acceptable. \\\n",
+    "You are provided with a conversation between a User and an Agent. Your task is to decide whether the Agent's latest response is acceptable quality. \\\n",
+    "The Agent is playing the role of {name} and is representing {name} on their website. \\\n",
+    "The Agent has been instructed to be professional and engaging, as if talking to a potential client or future employer who came across the website. \\\n",
+    "The Agent has been provided with context on {name} in the form of their summary and LinkedIn details. Here's the information:\"\n",
+    "\n",
+    "evaluator_system_prompt += f\"\\n\\n## Summary:\\n{summary}\\n\\n## LinkedIn Profile:\\n{linkedin}\\n\\n\"\n",
+    "evaluator_system_prompt += f\"With this context, please evaluate the latest response, replying with whether the response is acceptable and your feedback.\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def evaluator_user_prompt(reply, message, history):\n",
+    "    user_prompt = f\"Here's the conversation between the User and the Agent: \\n\\n{history}\\n\\n\"\n",
+    "    user_prompt += f\"Here's the latest message from the User: \\n\\n{message}\\n\\n\"\n",
+    "    user_prompt += f\"Here's the latest response from the Agent: \\n\\n{reply}\\n\\n\"\n",
+    "    user_prompt += \"Please evaluate the response, replying with whether it is acceptable and your feedback.\"\n",
+    "    return user_prompt"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "gemini = OpenAI(\n",
+    "    api_key=os.getenv(\"GOOGLE_API_KEY\"), \n",
+    "    base_url=\"https://generativelanguage.googleapis.com/v1beta/openai/\"\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def evaluate(reply, message, history) -> Evaluation:\n",
+    "\n",
+    "    messages = [{\"role\": \"system\", \"content\": evaluator_system_prompt}] + [{\"role\": \"user\", \"content\": evaluator_user_prompt(reply, message, history)}]\n",
+    "    response = gemini.beta.chat.completions.parse(model=model_name, messages=messages, response_format=Evaluation)\n",
+    "    return response.choices[0].message.parsed"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "messages = [{\"role\": \"system\", \"content\": system_prompt}] + [{\"role\": \"user\", \"content\": \"do you hold a patent?\"}]\n",
+    "response = gemini.chat.completions.create(model=model_name, messages=messages)\n",
+    "reply = response.choices[0].message.content"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "reply"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "evaluate(reply, \"do you hold a patent?\", messages[:1])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def rerun(reply, message, history, feedback):\n",
+    "    updated_system_prompt = system_prompt + \"\\n\\n## Previous answer rejected\\nYou just tried to reply, but the quality control rejected your reply\\n\"\n",
+    "    updated_system_prompt += f\"## Your attempted answer:\\n{reply}\\n\\n\"\n",
+    "    updated_system_prompt += f\"## Reason for rejection:\\n{feedback}\\n\\n\"\n",
+    "    messages = [{\"role\": \"system\", \"content\": updated_system_prompt}] + history + [{\"role\": \"user\", \"content\": message}]\n",
+    "    response = gemini.chat.completions.create(model=model_name, messages=messages)\n",
+    "    return response.choices[0].message.content"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def chat(message, history):\n",
+    "    if \"patent\" in message:\n",
+    "        system = system_prompt + \"\\n\\nEverything in your reply needs to be in pig latin - \\\n",
+    "              it is mandatory that you respond only and entirely in pig latin\"\n",
+    "    else:\n",
+    "        system = system_prompt\n",
+    "    messages = [{\"role\": \"system\", \"content\": system}] + history + [{\"role\": \"user\", \"content\": message}]\n",
+    "    response = gemini.chat.completions.create(model=model_name, messages=messages)\n",
+    "    reply =response.choices[0].message.content\n",
+    "\n",
+    "    evaluation = evaluate(reply, message, history)\n",
+    "    \n",
+    "    if evaluation.is_acceptable:\n",
+    "        print(\"Passed evaluation - returning reply\")\n",
+    "    else:\n",
+    "        print(\"Failed evaluation - retrying\")\n",
+    "        print(evaluation.feedback)\n",
+    "        reply = rerun(reply, message, history, evaluation.feedback)       \n",
+    "    return reply"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "gr.ChatInterface(chat, type=\"messages\").launch()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": ".venv",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.12.12"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}