From 04886f78d5f2b73e2646704fc9778f9e81ec6a82 Mon Sep 17 00:00:00 2001
From: Andrew Zhu <me@andrew-zhu.com>
Date: Wed, 19 Jul 2023 20:30:45 -0700
Subject: [PATCH] docs: add colab quickstart notebook

---
 examples/colab_quickstart.ipynb | 176 ++++++++++++++++++++++++++++++++
 1 file changed, 176 insertions(+)
 create mode 100644 examples/colab_quickstart.ipynb

diff --git a/examples/colab_quickstart.ipynb b/examples/colab_quickstart.ipynb
new file mode 100644
index 0000000..851165d
--- /dev/null
+++ b/examples/colab_quickstart.ipynb
@@ -0,0 +1,176 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+  "colab": {
+   "provenance": []
+  },
+  "kernelspec": {
+   "name": "python3",
+   "display_name": "Python 3"
+  },
+  "language_info": {
+   "name": "python"
+  }
+ },
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "source": [
+    "# Kani Quickstart\n",
+    "This colab notebook runs through the quickstart example found [here](https://github.com/zhudotexe/kani/blob/main/examples/1_quickstart.py).\n",
+    "\n",
+    "Feel free to make a copy of this notebook and modify the code cells to run other examples!"
+   ],
+   "metadata": {
+    "id": "eQIkAIRNMxOl"
+   }
+  },
+  {
+   "cell_type": "markdown",
+   "source": [
+    "## Install kani\n",
+    "First we install kani. kani requires Python 3.10+."
+   ],
+   "metadata": {
+    "id": "ae89bBMWM_fX"
+   }
+  },
+  {
+   "cell_type": "code",
+   "source": [
+    "!python --version\n",
+    "# for the latest development version:\n",
+    "# !pip install 'kani[openai] @ git+https://github.com/zhudotexe/kani.git@main'\n",
+    "# for the stable version:\n",
+    "!pip install -qq 'kani[openai]'"
+   ],
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "id": "dzJhGTAYLneA",
+    "outputId": "9c8ec193-4ff1-48ef-b894-a09015367634"
+   },
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "markdown",
+   "source": [
+    "## Imports\n",
+    "Then, import all the necessary components."
+   ],
+   "metadata": {
+    "id": "j4kWNrq8NGQN"
+   }
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/",
+     "height": 374
+    },
+    "id": "g8gn1WknKsSD",
+    "outputId": "151aa67b-18dd-4216-a678-b6f1a3235397"
+   },
+   "outputs": [],
+   "source": [
+    "from kani import Kani, chat_in_terminal_async\n",
+    "from kani.engines.openai import OpenAIEngine"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "source": [
+    "## OpenAI Key\n",
+    "To use the OpenAIEngine, you need your OpenAI API key. You can find it here: https://platform.openai.com/account/api-keys"
+   ],
+   "metadata": {
+    "id": "k9m_ytjXP_xy"
+   }
+  },
+  {
+   "cell_type": "code",
+   "source": [
+    "# Insert your OpenAI API key (https://platform.openai.com/account/api-keys)\n",
+    "api_key = \"sk-...\"  #@param {type:\"string\"}"
+   ],
+   "metadata": {
+    "cellView": "form",
+    "id": "QSP2oODLLWwL"
+   },
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "markdown",
+   "source": [
+    "## Kani\n",
+    "Set up the kani engine and harness.\n",
+    "\n",
+    "kani uses an Engine to interact with the language model. You can specify other model parameters in the engine, like `temperature=0.7`, or change the model here.\n",
+    "\n",
+    "You can also try uncommenting the LLaMA code and using the LLaMA v2 engine! You'll need to supply your huggingface hub token (and likely change the Colab runtime to an A100 GPU runtime).\n",
+    "\n",
+    "The kani manages the chat state, prompting, and function calling. Here, we only give it the engine to call\n",
+    "ChatGPT, but you can specify other parameters like `system_prompt=\"You are...\"` in the kani."
+   ],
+   "metadata": {
+    "id": "13re5suiQL2f"
+   }
+  },
+  {
+   "cell_type": "code",
+   "source": [
+    "# uncomment the next 4 lines to use LLaMA\n",
+    "# !pip install -q 'kani[huggingface,llama]'\n",
+    "# hf_token = \"...\"  # insert your huggingface hub token here\n",
+    "# from kani.engines.huggingface.llama2 import LlamaEngine\n",
+    "# engine = LlamaEngine(use_auth_token=hf_token, strict=True)\n",
+    "\n",
+    "# comment the next line if using LLaMA\n",
+    "engine = OpenAIEngine(api_key, model=\"gpt-3.5-turbo\")\n",
+    "ai = Kani(engine)"
+   ],
+   "metadata": {
+    "id": "Cb_fBh4JLY_5"
+   },
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "markdown",
+   "source": [
+    "## Chat\n",
+    "kani comes with a utility to interact with a kani through your terminal! Since we're in a colab environment, we need to use `chat_in_terminal_async`.\n",
+    "\n",
+    "You can end the chat by sending the message `!stop`.\n",
+    "\n",
+    "Check out the docs for how to use kani programmatically."
+   ],
+   "metadata": {
+    "id": "3j0UOmxCQgwC"
+   }
+  },
+  {
+   "cell_type": "code",
+   "source": [
+    "# google colab may complain about \"await only allowed within async function\"\n",
+    "# however, the colab interpreter is an async environment, so this is ok\n",
+    "await chat_in_terminal_async(ai, stopword=\"!stop\")"
+   ],
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "id": "I_FShlgILcmo",
+    "outputId": "761f5f84-b83a-4097-8266-ed6f97aeff07"
+   },
+   "execution_count": null,
+   "outputs": []
+  }
+ ]
+}