gpt/nlp_with_gpt_notebook.ipynb

{"cells":[{"cell_type":"markdown","metadata":{"id":"EV0cfuXJeO-8"},"source":["<a href=\"https://colab.research.google.com/github/ml4devs/ml4devs-notebooks/blob/master/gpt/nlp_with_gpt_notebook.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"]},{"cell_type":"markdown","metadata":{"id":"e3Id4cfBeO-_"},"source":["<h1><center>Common NLP Tasks with GPT: Sentiment Analysis, Language Translation, and Named-Entity Recognition</center></h1>\n","\n","<p><center>\n","<address>&copy; Satish Chandra Gupta<br/>\n","LinkedIn: <a href=\"https://www.linkedin.com/in/scgupta/\">scgupta</a>,\n","Twitter: <a href=\"https://twitter.com/scgupta\">scgupta</a>\n","</address>\n","</center></p>\n","\n","---\n","\n","## Setup\n","\n","### Install Pip Packages\n","\n","You need Python 3.7 or higher to install [OpenAI Python API library](https://github.com/openai/openai-python)."]},{"cell_type":"code","execution_count":1,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":770,"status":"ok","timestamp":1701355793267,"user":{"displayName":"","userId":""},"user_tz":-330},"id":"iZkQzDDxeO_A","outputId":"4824a891-78f6-47d5-c78b-5f51c7566483","vscode":{"languageId":"shellscript"}},"outputs":[{"name":"stdout","output_type":"stream","text":["Python 3.10.14\n"]}],"source":["# You should have Python 3.7 or higher\n","\n","!python --version\n"]},{"cell_type":"code","execution_count":2,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":5781,"status":"ok","timestamp":1701355799502,"user":{"displayName":"","userId":""},"user_tz":-330},"id":"1Y6r5ieIeO_C","outputId":"c6a9c2b8-42a5-4749-d018-df48a804223e","vscode":{"languageId":"shellscript"}},"outputs":[],"source":["!pip install openai python-dotenv &> /dev/null\n"]},{"cell_type":"markdown","metadata":{"id":"cYfbkKGPeO_D"},"source":["### Upload `.env` File with API Keys\n","\n","You can either use GPT directly from OpenAI, or you can use Azure OpenAI from Microsoft. You need to create a `.env` file and add the environment variables needed for OpenAI api.\n","\n","If you are using OpenAI, check your [OpenAI account](https://platform.openai.com/api-keys) for creating API key. Your `.env` file will look like following:\n","\n","```sh\n","$ cat .env\n","OPENAI_API_KEY='sk-YourOpenAiApiKeyHere'\n","```\n","\n","If you are using Microsoft Azure OpenAI:\n","- Go to [Azure Portal](https://portal.azure.com/) > **All Resources**\n","- Filter the list with Type == Azure OpenAI\n","- Select the one you plan to use\n","- If there are none, you can [create and deploy an Azure OpenAI Service resource](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/create-resource?pivots=web-portal#create-a-resource)\n","- Click on **Keys and Endpoint** on the left menu\n","- Get `AZURE_OPENAI_API_KEY` and `AZURE_OPENAI_ENDPOINT`\n","- Next click **Model deployments** on the left menu, and then click **Manage Deployment** button\n","- Alternatively, you can go to [Azure OpenAI Studio](https://oai.azure.com/), and click **Deployments** on the left menu\n","- Find (the latest) API version for [Azure OpenAI Service](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#rest-api-versioning)\n","\n","Your `.env` file will look like following:\n","```sh\n","$ cat .env\n","AZURE_OPENAI_API_KEY=yourAzureOpenAiApiKey\n","AZURE_OPENAI_ENDPOINT=https://your-azure-deployment.openai.azure.com/\n","AZURE_OPENAI_DEPLOYMENT_ID=your-deployment-name\n","AZURE_OPENAI_API_VERSION=2023-10-01-preview\n","```\n","\n","Upload `.env` using Upload File button in Google Colab (or Jupyter Notebook). In worst case scenario, uncomment and modify the relevant lines in the following cell to create `.env` file. Please note that it is dangerous to share such notebooks or check them into git."]},{"cell_type":"code","execution_count":3,"metadata":{"id":"XKy_X1wdeO_D","vscode":{"languageId":"shellscript"}},"outputs":[],"source":["# Upload or create a .env file with (Azure) OpenAI API creds\n","\n","#!echo \"OPENAI_API_KEY=sk-YourOpenApiKeyHere\" >> .env\n","\n","#!echo \"AZURE_OPENAI_API_KEY=yourAzureOpenAiApiKey\" >> .env\n","#!echo \"AZURE_OPENAI_ENDPOINT=https://your-azure-deployment.openai.azure.com/\" >> .env\n","#!echo \"AZURE_OPENAI_DEPLOYMENT_ID=your-deployment-name\" >> .env\n","#!echo \"AZURE_OPENAI_API_VERSION=2023-10-01-preview\" >> .env\n"]},{"cell_type":"markdown","metadata":{"id":"KcPWLtAWeO_E"},"source":["### Load `.env` File and Specify (Azure) OpenAI GPT Model\n","\n","Load environment variables from `.env` file:"]},{"cell_type":"code","execution_count":4,"metadata":{"id":"tIT35MxGeO_F"},"outputs":[],"source":["from dotenv import load_dotenv, find_dotenv\n","\n","_ = load_dotenv(find_dotenv())\n"]},{"cell_type":"markdown","metadata":{"id":"cVN8Z7wQeO_F"},"source":["Set `IS_AZURE_OPENAI` flag to `True`, if you are using Azure OpenAI:"]},{"cell_type":"code","execution_count":5,"metadata":{"id":"KfXGuwQZeO_G"},"outputs":[],"source":["IS_AZURE_OPENAI: bool = False\n"]},{"cell_type":"markdown","metadata":{"id":"2mY86GV2eO_G"},"source":["Specify model name:"]},{"cell_type":"code","execution_count":6,"metadata":{"id":"GK1r2VFVeO_G"},"outputs":[],"source":["from datetime import datetime\n","\n","GPT35_TURBO: str = \"gpt-3.5-turbo-1106\" if datetime.now() < datetime(2023, 12, 11) else \"gpt-3.5-turbo\"\n"]},{"cell_type":"markdown","metadata":{"id":"87QWD_kLeO_G"},"source":["---\n","\n","## Create an OpenAI Client and Specify GPT Model"]},{"cell_type":"code","execution_count":7,"metadata":{"id":"J3gYa6czeO_H"},"outputs":[],"source":["import os\n","import openai\n"]},{"cell_type":"code","execution_count":8,"metadata":{"id":"yTp0GKx9eO_H"},"outputs":[],"source":["def create_open_ai_client():\n","    if IS_AZURE_OPENAI:\n","        return openai.AzureOpenAI(\n","            api_key=os.getenv(\"AZURE_OPENAI_API_KEY\"),\n","            api_version=os.getenv(\"AZURE_OPENAI_API_VERSION\"),\n","            azure_endpoint=os.getenv(\"AZURE_OPENAI_ENDPOINT\"),\n","            azure_deployment=os.getenv(\"AZURE_OPENAI_DEPLOYMENT_ID\")\n","        )\n","    else:\n","        return openai.OpenAI(\n","            api_key=os.getenv('OPENAI_API_KEY')\n","        )\n"]},{"cell_type":"code","execution_count":9,"metadata":{"id":"Hes3w6mfeO_H"},"outputs":[],"source":["openai_client = create_open_ai_client()\n","openai_model = os.getenv(\"AZURE_OPENAI_DEPLOYMENT_ID\") if IS_AZURE_OPENAI else GPT35_TURBO\n","\n","def get_gpt_response(prompt, model=openai_model, temperature=0):\n","    messages = [{\"role\": \"user\", \"content\": prompt}]\n","    response = openai_client.chat.completions.create(\n","        model=model,\n","        #response_format={\"type\": \"json_object\"},  # Uncomment it if your chosen model supports it\n","        messages=messages,\n","        temperature=temperature,\n","    )\n","    return response.choices[0].message.content\n"]},{"cell_type":"code","execution_count":10,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":831,"status":"ok","timestamp":1701355800331,"user":{"displayName":"","userId":""},"user_tz":-330},"id":"5JrKbCv7eO_H","outputId":"0202742f-4049-4f17-f581-9d2508d021c8"},"outputs":[{"name":"stdout","output_type":"stream","text":["{\n","  \"message\": \"This is a test\"\n","}\n"]}],"source":["print(get_gpt_response(\"Say this is test in JSON\"))\n"]},{"cell_type":"markdown","metadata":{"id":"Mu09cyaLeO_I"},"source":["You are all set to use GPT for common NLP tasks such as Sentiment Analysis, Language Translation, Intent/Entity Recognition."]},{"cell_type":"markdown","metadata":{"id":"eOmv1Y7OeO_I"},"source":["---\n","\n","## Sentiment Analysis\n","\n","Let's do sentiment analysis for food reviews. In classical ML, you will need to build a supervised classification model for sentiment analysis. You need to:\n","\n","- Clean and label the data (this takes significant amount of effort)\n","- Divide it into train, validate, and test sets\n","- Preprocessing: remove stop words, stemming, etc.\n","- Train multiple models\n","- Measure inference accuracy\n","- Select a model, and tune its hyper-parameters\n","- Deploy the final model\n","\n","This whole endeavour may take a couple of weeks and sometime months!\n","\n","But Large Language Models (LLMs) like GPT eliminates ML model training or train it with just few examples. It is called [Zero or Few Shot Learning](https://en.wikipedia.org/wiki/Zero-shot_learning). This is because foundational LLM models are capable of doing multiple tasks.\n","\n","This effectively makes many NLP capabilities accessible to developers who may not have data science and machine learning expertise. And, they can do it in few hours or days (instead of weeks and months)!\n","\n","See it yourself. Here is your food review sentiment analyzer with few lines of code."]},{"cell_type":"code","execution_count":11,"metadata":{"id":"CTkZo8K7eO_I"},"outputs":[],"source":["food_reviews = [\n","    \"The food is great, ambience is just right, but service is slow.\",\n","    \"खाना बहुत स्वादिष्ट है, बैंगन भरता और काबुली चिकन कबाब जरूर खाएँ\",\n","    \"starters soggy and लस्सी बिलकुल पानी, बकवास खाना, waste of money\",\n","]\n"]},{"cell_type":"code","execution_count":12,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":971,"status":"ok","timestamp":1701355801300,"user":{"displayName":"","userId":""},"user_tz":-330},"id":"FsiUsSlXeO_I","outputId":"f8eee039-b029-4552-f5df-51604627f1af"},"outputs":[{"name":"stdout","output_type":"stream","text":["{\n","  \"sentiment\": \"mixed\"\n","}\n"]}],"source":["prompt = f\"\"\"\n","What is the sentiment of the following review that is delimited with triple backticks?\n","\n","Format your response in JSON.\n","\n","Review text: ```{food_reviews[0]}```\n","\"\"\"\n","\n","print(get_gpt_response(prompt))\n"]},{"cell_type":"markdown","metadata":{"id":"ClX5SA-reO_J"},"source":["Voilà! It worked like a charm! With just 20-word long prompt! Now let's improve the prompt to get the response in a structure that you specify."]},{"cell_type":"code","execution_count":13,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":1749,"status":"ok","timestamp":1701355803046,"user":{"displayName":"","userId":""},"user_tz":-330},"id":"n6BXbiIAeO_J","outputId":"d4cdea6f-805f-498e-f438-04e5442a451f"},"outputs":[{"name":"stdout","output_type":"stream","text":["{\n","    \"sentiment\": \"mixed\",\n","    \"stars\": 3,\n","    \"emotions\": [\"positive\", \"neutral\"],\n","    \"summary\": \"Great food and ambience, but slow service.\"\n","}\n"]}],"source":["prompt = f\"\"\"\n","Identify following items from the review text that is delimited with triple backticks:\n","- Sentiment: (positive, mixed, or negative)\n","- Stars: a number rating characterizing overall sentiment, 1 star being the lowest and 5 star being the highest\n","- Emotions: top emotion(s), maximum 3 emotions\n","- Summary: human readable summary of the review and sentiments in less than 255 characters\n","\n","Format your response as JSON with \"sentiment\", \"stars\", \"emotions\", and \"summary\" as the keys.\n","\n","Review text: ```{food_reviews[0]}```\n","\"\"\"\n","\n","print(get_gpt_response(prompt))\n"]},{"cell_type":"markdown","metadata":{"id":"GzjaaFRkeO_J"},"source":["---\n","\n","## Language Translation\n","\n","Now, let's make this sentiment analyzer multi-lingual.\n","\n","GPT has language identification and translation capabilities, and you can invoke them with a simple prompt."]},{"cell_type":"code","execution_count":14,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":2852,"status":"ok","timestamp":1701355805897,"user":{"displayName":"","userId":""},"user_tz":-330},"id":"qCMoL5CteO_J","outputId":"b99460b9-a078-482e-82e8-a80a61c3a0ff"},"outputs":[{"name":"stdout","output_type":"stream","text":["{\n","    \"text\": \"खाना बहुत स्वादिष्ट है, बैंगन भरता और काबुली चिकन कबाब जरूर खाएँ\",\n","    \"language\": \"Hindi\",\n","    \"translation\": \"The food is very delicious, be sure to try Baingan Bharta and Kabuli Chicken Kebab\"\n","}\n"]}],"source":["prompt = f\"\"\"\n","Translate the input text into English.\n","\n","Format your response as JSON with values for following keys:\n","- text: input text as is\n","- language: the language of the input text\n","- translation: input text translated in English\n","\n","Input text: ```{food_reviews[1]}```\n","\"\"\"\n","\n","print(get_gpt_response(prompt))\n"]},{"cell_type":"markdown","metadata":{"id":"FljOMrZqeO_J"},"source":["You can see the pattern:\n","- Breakdown the task into smaller steps\n","- Give specific instructions for each step\n","- Include input with clear demarcation\n","- Specify the desired structure of the output\n","\n","It is almost like how you will teach a smart kid to do a specific task.\n","\n","You can change the functionality by changing the prompt. You can experiment and craft an effective prompt for your NLP task.\n","\n","Now let's put together sentiment analysis and translation."]},{"cell_type":"code","execution_count":15,"metadata":{"id":"bCCDancUeO_J"},"outputs":[],"source":["def infer_sentiment(text):\n","    prompt = f\"\"\"\n","        Identify following items from the review text:\n","        - Language: language of the review text\n","        - Translation: review text translated in English\n","        - Sentiment: (positive, mixed, or negative)\n","        - Stars: a number rating characterizing overall sentiment, 1 star being the lowest and 5 star being the highest\n","        - Emotions: top emotion(s), maximum 3 emotions\n","        - Summary: human readable summary of the review and sentiments in less than 255 characters\n","\n","        Format your response as JSON with \"language\", \"translation\", \"sentiment\", \"stars\", \"emotions\", and summary as the keys.\n","\n","        Review text: '''{text}'''\n","    \"\"\"\n","\n","    return get_gpt_response(prompt)\n"]},{"cell_type":"code","execution_count":16,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":8417,"status":"ok","timestamp":1701355814312,"user":{"displayName":"","userId":""},"user_tz":-330},"id":"xsrC5eoneO_K","outputId":"344be55d-2164-4353-eace-726ab9459a1f"},"outputs":[{"name":"stdout","output_type":"stream","text":["{\n","    \"language\": \"English\",\n","    \"translation\": \"The food is great, ambience is just right, but service is slow.\",\n","    \"sentiment\": \"mixed\",\n","    \"stars\": 3,\n","    \"emotions\": [\"satisfaction\", \"disappointment\"],\n","    \"summary\": \"Enjoyable experience with great food and ambience, but service could be improved.\"\n","}\n","{\n","    \"language\": \"Hindi\",\n","    \"translation\": \"The food is very delicious, do try the stuffed eggplant and Kabul chicken kebab\",\n","    \"sentiment\": \"positive\",\n","    \"stars\": 5,\n","    \"emotions\": [\"delight\", \"satisfaction\"],\n","    \"summary\": \"Delicious food with must-try stuffed eggplant and Kabul chicken kebab, highly satisfying experience with a 5-star rating\"\n","}\n","{\n","    \"language\": \"English\",\n","    \"translation\": \"starters soggy and lassi completely watery, terrible food, waste of money\",\n","    \"sentiment\": \"negative\",\n","    \"stars\": 1,\n","    \"emotions\": [\"disappointment\", \"disgust\"],\n","    \"summary\": \"Extremely disappointing experience with watery lassi and soggy starters, terrible food quality, complete waste of money.\"\n","}\n"]}],"source":["for t in food_reviews:\n","    print(infer_sentiment(t))\n"]},{"cell_type":"markdown","metadata":{"id":"i7yG6NszeO_K"},"source":["---\n","\n","## Intent/Entity Extraction"]},{"cell_type":"markdown","metadata":{"id":"L2Rv8bKpeO_K"},"source":["[Named-Entity Recognition (NER)](https://en.wikipedia.org/wiki/Named-entity_recognition) is another very common NLP task. For example, Chatbots and Voice Assistants have to:\n","\n","- Infer what you want (intent)\n","- Extract the named entities from your sentences that are needed to fulfill your request\n","- Perform that request\n","\n","For example, each of these commands to Alexa have different intent, and entities associated with it:\n","- Play songs by Taylor Swift\n","- Set an alarm for 30 minutes\n","- How is the weather\n","\n","Let's build a multilingual intent/entity extractor for a travel assistant that can enquire, book, and cancel bus, train, and flight tickets."]},{"cell_type":"code","execution_count":17,"metadata":{"id":"7GTL-q4xeO_K"},"outputs":[],"source":["travel_messages = [\n","    \"I want to fly from Bangalore to Delhi\",\n","    \"मुझे कल कानपुर से लखनऊ के लिए बस टिकट बुक करना है\",\n","    \"ನನ್ನ ಬಸ್ ಟಿಕೆಟ್ ರದ್ದು ಮಾಡಿ\",\n","]\n"]},{"cell_type":"code","execution_count":18,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":2483,"status":"ok","timestamp":1701355816794,"user":{"displayName":"","userId":""},"user_tz":-330},"id":"rhQIzyhWeO_K","outputId":"96f23626-b6ad-44f8-d6d8-cadd8c945a9b"},"outputs":[{"name":"stdout","output_type":"stream","text":["{\n","  \"language\": \"Hindi\",\n","  \"translation\": \"I want to book a bus ticket from Kanpur to Lucknow for tomorrow\",\n","  \"intent\": \"book\",\n","  \"mode\": \"bus\",\n","  \"date\": \"2023-10-25\",\n","  \"source\": \"Kanpur\",\n","  \"destination\": \"Lucknow\"\n","}\n"]}],"source":["prompt = f\"\"\"\n","Act as a travel assistant clerk. Your job is to help customers by bus, train, or flight.\n","Identify following items from a customer message:\n","- Language: language of the customer message\n","- Translation: customer message translated in English\n","- Intent: (inquire, book, or cancel)\n","- Mode: (bus, train, or flight)\n","- Date: the travel date in YYYY-MM-DD\n","- Source: starting place of the journey\n","If the information isn't present, use null as the value.\n","\n","Format your response as JSON with \"language\", \"translation\", \"intent\", \"mode\", \"source\", and \"destination\".\n","\n","Review test: '''{travel_messages[1]}'''\n","\"\"\"\n","\n","print(get_gpt_response(prompt))\n"]},{"cell_type":"markdown","metadata":{"id":"sgzgjdvveO_L"},"source":["Well, it almost got everything right, except the date. It inferred \"tomorrow\" incorrectly, maybe because \"today\" for the model is when it was trained or deployed.\n","\n","That is another important lesson: your prompt must have the needed context. Let's tell it what the date today is."]},{"cell_type":"code","execution_count":19,"metadata":{"id":"HqsgF3g3eO_L"},"outputs":[],"source":["def travel_assistant(text):\n","    prompt = f\"\"\"\n","       Act as a travel assistant clerk. Your job is to help customers by bus, train, or flight.\n","       Identify following items from a customer message:\n","       - Language: language of the customer message\n","       - Translation: customer message translated in English\n","       - Intent: (inquire, book, or cancel)\n","       - Mode: (bus, train, or flight)\n","       - Date: the travel date in YYYY-MM-DD\n","       - Source: starting place of the journey\n","       If the information isn't present, use null as the value.\n","\n","       The current date and time is {datetime.now().strftime(\"%d %b %Y %I:%M %p\")}\n","\n","       Format your response as JSON with \"language\", \"translation\", \"intent\", \"mode\", \"source\", and \"destination\".\n","\n","       Review test: '''{text}'''\n","    \"\"\"\n","\n","    return get_gpt_response(prompt)\n"]},{"cell_type":"code","execution_count":20,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":6788,"status":"ok","timestamp":1701355823568,"user":{"displayName":"","userId":""},"user_tz":-330},"id":"6UyObQVfeO_L","outputId":"e3815064-3811-48ea-d654-971e55df4ccc"},"outputs":[{"name":"stdout","output_type":"stream","text":["{\n","    \"language\": \"English\",\n","    \"translation\": \"I want to fly from Bangalore to Delhi\",\n","    \"intent\": \"book\",\n","    \"mode\": \"flight\",\n","    \"date\": null,\n","    \"source\": \"Bangalore\",\n","    \"destination\": \"Delhi\"\n","}\n","{\n","    \"language\": \"Hindi\",\n","    \"translation\": \"I want to book a bus ticket from Kanpur to Lucknow tomorrow\",\n","    \"intent\": \"book\",\n","    \"mode\": \"bus\",\n","    \"date\": \"2024-10-25\",\n","    \"source\": \"Kanpur\",\n","    \"destination\": \"Lucknow\"\n","}\n","{\n","    \"language\": \"Kannada\",\n","    \"translation\": \"Cancel my bus ticket\",\n","    \"intent\": \"cancel\",\n","    \"mode\": \"bus\",\n","    \"date\": null,\n","    \"source\": null,\n","    \"destination\": null\n","}\n"]}],"source":["for t in travel_messages:\n","    print(travel_assistant(t))\n"]},{"cell_type":"markdown","metadata":{"id":"f5BQT17FeO_L"},"source":["LLMs are one of the most powerful models, and yet most accessible for developers. It reduces the time to experiment, prototype, and deploy sophisticated NLP-assisted applications."]},{"cell_type":"markdown","metadata":{"id":"-_oyoZAkeO_L"},"source":["---\n","<p>Copyright &copy 2023 <a href=\"https://www.linkedin.com/in/scgupta\">Satish Chandra Gupta</a>.</p>\n","<img src=\"https://licensebuttons.net/l/by-nc-sa/3.0/88x31.png\" align=\"left\"/> <p>&nbsp;<a href=\"https://creativecommons.org/licenses/by-nc-sa/4.0/\">CC BY-NC-SA 4.0 International</a> License.</p>"]}],"metadata":{"colab":{"provenance":[{"file_id":"https://github.com/ml4devs/ml4devs-notebooks/blob/master/gpt/nlp_with_gpt_notebook.ipynb","timestamp":1701355922767},{"file_id":"https://github.com/ml4devs/ml4devs-notebooks/blob/master/gpt/nlp_with_gpt_notebook.ipynb","timestamp":1701352821148}]},"kernelspec":{"display_name":"kaggle","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.14"}},"nbformat":4,"nbformat_minor":0}