-
-
Notifications
You must be signed in to change notification settings - Fork 27
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
1 changed file
with
176 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,176 @@ | ||
{ | ||
"nbformat": 4, | ||
"nbformat_minor": 0, | ||
"metadata": { | ||
"colab": { | ||
"provenance": [] | ||
}, | ||
"kernelspec": { | ||
"name": "python3", | ||
"display_name": "Python 3" | ||
}, | ||
"language_info": { | ||
"name": "python" | ||
} | ||
}, | ||
"cells": [ | ||
{ | ||
"cell_type": "markdown", | ||
"source": [ | ||
"# Kani Quickstart\n", | ||
"This colab notebook runs through the quickstart example found [here](https://github.com/zhudotexe/kani/blob/main/examples/1_quickstart.py).\n", | ||
"\n", | ||
"Feel free to make a copy of this notebook and modify the code cells to run other examples!" | ||
], | ||
"metadata": { | ||
"id": "eQIkAIRNMxOl" | ||
} | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"source": [ | ||
"## Install kani\n", | ||
"First we install kani. kani requires Python 3.10+." | ||
], | ||
"metadata": { | ||
"id": "ae89bBMWM_fX" | ||
} | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"source": [ | ||
"!python --version\n", | ||
"# for the latest development version:\n", | ||
"# !pip install 'kani[openai] @ git+https://github.com/zhudotexe/kani.git@main'\n", | ||
"# for the stable version:\n", | ||
"!pip install -qq 'kani[openai]'" | ||
], | ||
"metadata": { | ||
"colab": { | ||
"base_uri": "https://localhost:8080/" | ||
}, | ||
"id": "dzJhGTAYLneA", | ||
"outputId": "9c8ec193-4ff1-48ef-b894-a09015367634" | ||
}, | ||
"execution_count": null, | ||
"outputs": [] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"source": [ | ||
"## Imports\n", | ||
"Then, import all the necessary components." | ||
], | ||
"metadata": { | ||
"id": "j4kWNrq8NGQN" | ||
} | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"metadata": { | ||
"colab": { | ||
"base_uri": "https://localhost:8080/", | ||
"height": 374 | ||
}, | ||
"id": "g8gn1WknKsSD", | ||
"outputId": "151aa67b-18dd-4216-a678-b6f1a3235397" | ||
}, | ||
"outputs": [], | ||
"source": [ | ||
"from kani import Kani, chat_in_terminal_async\n", | ||
"from kani.engines.openai import OpenAIEngine" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"source": [ | ||
"## OpenAI Key\n", | ||
"To use the OpenAIEngine, you need your OpenAI API key. You can find it here: https://platform.openai.com/account/api-keys" | ||
], | ||
"metadata": { | ||
"id": "k9m_ytjXP_xy" | ||
} | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"source": [ | ||
"# Insert your OpenAI API key (https://platform.openai.com/account/api-keys)\n", | ||
"api_key = \"sk-...\" #@param {type:\"string\"}" | ||
], | ||
"metadata": { | ||
"cellView": "form", | ||
"id": "QSP2oODLLWwL" | ||
}, | ||
"execution_count": null, | ||
"outputs": [] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"source": [ | ||
"## Kani\n", | ||
"Set up the kani engine and harness.\n", | ||
"\n", | ||
"kani uses an Engine to interact with the language model. You can specify other model parameters in the engine, like `temperature=0.7`, or change the model here.\n", | ||
"\n", | ||
"You can also try uncommenting the LLaMA code and using the LLaMA v2 engine! You'll need to supply your huggingface hub token (and likely change the Colab runtime to an A100 GPU runtime).\n", | ||
"\n", | ||
"The kani manages the chat state, prompting, and function calling. Here, we only give it the engine to call\n", | ||
"ChatGPT, but you can specify other parameters like `system_prompt=\"You are...\"` in the kani." | ||
], | ||
"metadata": { | ||
"id": "13re5suiQL2f" | ||
} | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"source": [ | ||
"# uncomment the next 4 lines to use LLaMA\n", | ||
"# !pip install -q 'kani[huggingface,llama]'\n", | ||
"# hf_token = \"...\" # insert your huggingface hub token here\n", | ||
"# from kani.engines.huggingface.llama2 import LlamaEngine\n", | ||
"# engine = LlamaEngine(use_auth_token=hf_token, strict=True)\n", | ||
"\n", | ||
"# comment the next line if using LLaMA\n", | ||
"engine = OpenAIEngine(api_key, model=\"gpt-3.5-turbo\")\n", | ||
"ai = Kani(engine)" | ||
], | ||
"metadata": { | ||
"id": "Cb_fBh4JLY_5" | ||
}, | ||
"execution_count": null, | ||
"outputs": [] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"source": [ | ||
"## Chat\n", | ||
"kani comes with a utility to interact with a kani through your terminal! Since we're in a colab environment, we need to use `chat_in_terminal_async`.\n", | ||
"\n", | ||
"You can end the chat by sending the message `!stop`.\n", | ||
"\n", | ||
"Check out the docs for how to use kani programmatically." | ||
], | ||
"metadata": { | ||
"id": "3j0UOmxCQgwC" | ||
} | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"source": [ | ||
"# google colab may complain about \"await only allowed within async function\"\n", | ||
"# however, the colab interpreter is an async environment, so this is ok\n", | ||
"await chat_in_terminal_async(ai, stopword=\"!stop\")" | ||
], | ||
"metadata": { | ||
"colab": { | ||
"base_uri": "https://localhost:8080/" | ||
}, | ||
"id": "I_FShlgILcmo", | ||
"outputId": "761f5f84-b83a-4097-8266-ed6f97aeff07" | ||
}, | ||
"execution_count": null, | ||
"outputs": [] | ||
} | ||
] | ||
} |