Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix "cache_control": {"type": "ephemeral"} #121

Closed
wants to merge 1 commit into from
+12 −145
Closed

Fix "cache_control": {"type": "ephemeral"} #121

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
157 changes: 12 additions & 145 deletions misc/prompt_caching.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,17 +28,9 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Note: you may need to restart the kernel to use updated packages.\n"
]
}
],
"outputs": [],
"source": [
"%pip install anthropic bs4 --quiet"
]
Expand Down Expand Up @@ -69,25 +61,7 @@
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Fetched 737522 characters from the book.\n",
"First 500 characters:\n",
"The Project Gutenberg eBook of Pride and Prejudice\n",
"This ebook is for the use of anyone anywhere in the United States and\n",
"most other parts of the world at no cost and with almost no restrictions\n",
"whatsoever. You may copy it, give it away or re-use it under the terms\n",
"of the Project Gutenberg License included with this ebook or online\n",
"at www.gutenberg.org. If you are not located in the United States,\n",
"you will have to check the laws of the country where you are located\n",
"before using this eBook.\n",
"Title:\n"
]
}
],
"outputs": [],
"source": [
"def fetch_article_content(url):\n",
" response = requests.get(url)\n",
Expand Down Expand Up @@ -140,22 +114,9 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Non-cached API call time: 20.37 seconds\n",
"Non-cached API call input tokens: 17\n",
"Non-cached API call output tokens: 8\n",
"\n",
"Summary (non-cached):\n",
"[TextBlock(text='Pride and Prejudice', type='text')]\n"
]
}
],
"outputs": [],
"source": [
"def make_non_cached_api_call():\n",
" messages = [\n",
Expand All @@ -164,8 +125,7 @@
" \"content\": [\n",
" {\n",
" \"type\": \"text\",\n",
" \"text\": \"<book>\" + book_content + \"</book>\",\n",
" \"cache_control\": {\"type\": \"ephemeral\"}\n",
" \"text\": \"<book>\" + book_content + \"</book>\"\n",
" },\n",
" {\n",
" \"type\": \"text\",\n",
Expand Down Expand Up @@ -210,22 +170,9 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Cached API call time: 2.92 seconds\n",
"Cached API call input tokens: 17\n",
"Cached API call output tokens: 8\n",
"\n",
"Summary (cached):\n",
"[TextBlock(text='Pride and Prejudice', type='text')]\n"
]
}
],
"outputs": [],
"source": [
"def make_cached_api_call():\n",
" messages = [\n",
Expand Down Expand Up @@ -284,89 +231,9 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Turn 1:\n",
"User: What is the title of this novel?\n",
"Assistant: The title of this novel is \"Pride and Prejudice\" by Jane Austen.\n",
"User input tokens: 4\n",
"Output tokens: 22\n",
"Input tokens (cache read): 0\n",
"Input tokens (cache write): 187354\n",
"0.0% of input prompt cached (4 tokens)\n",
"Time taken: 22.04 seconds\n",
"\n",
"Turn 2:\n",
"User: Who are Mr. and Mrs. Bennet?\n",
"Assistant: Mr. and Mrs. Bennet are the parents of five daughters (Jane, Elizabeth, Mary, Kitty, and Lydia) in Pride and Prejudice. Their personalities are quite different from each other:\n",
"\n",
"Mr. Bennet is an intelligent but rather detached and sarcastic man who often retreats to his library to avoid his wife's dramatic behavior. He is fond of his daughter Elizabeth (Lizzy) in particular due to her wit and good sense. While he can be amusing, he is also somewhat negligent in his duties as a father and head of the household.\n",
"\n",
"Mrs. Bennet is a rather foolish, nervous, and obsessive woman whose main goal in life is to see her daughters married to wealthy men. This preoccupation stems partly from the fact that their estate is entailed to a male heir (Mr. Collins), meaning her daughters will be left with little after Mr. Bennet's death. She often embarrasses her more sensible daughters (particularly Elizabeth and Jane) with her lack of social graces and her obvious attempts at matchmaking.\n",
"\n",
"Their marriage is portrayed as an unfortunate example of a match based on superficial attraction (Mr. Bennet was attracted to Mrs. Bennet's youth and beauty) rather than compatibility of character and intellect, serving as a contrast to the more meaningful relationships that develop later in the novel.\n",
"User input tokens: 4\n",
"Output tokens: 295\n",
"Input tokens (cache read): 187354\n",
"Input tokens (cache write): 36\n",
"100.0% of input prompt cached (187358 tokens)\n",
"Time taken: 8.32 seconds\n",
"\n",
"Turn 3:\n",
"User: What is Netherfield Park?\n",
"Assistant: Netherfield Park is a large estate near the Bennets' home of Longbourn that is rented by Mr. Bingley at the beginning of the novel. It becomes a central location in the story as Mr. Bingley's arrival there sets many of the novel's key events in motion.\n",
"\n",
"The arrival of Mr. Bingley at Netherfield creates much excitement in the neighborhood, particularly for Mrs. Bennet, who sees him as a potential husband for one of her daughters due to his wealth (he has an income of around 4-5 thousand pounds a year). Indeed, it is at Netherfield where the romance between Jane Bennet and Mr. Bingley begins to develop.\n",
"\n",
"Netherfield is also significant because it is where Jane falls ill while visiting Mr. Bingley's sisters, leading to Elizabeth coming to nurse her. This extended stay at Netherfield provides important opportunities for Elizabeth to interact with Mr. Darcy, though at this point in the novel their interactions are marked by pride and prejudice on both sides.\n",
"\n",
"The house and estate represent wealth and social status in the novel, and its occupancy by the Bingleys brings the higher social circles of London into contact with the more provincial society of Meryton and its surroundings.\n",
"User input tokens: 4\n",
"Output tokens: 286\n",
"Input tokens (cache read): 187390\n",
"Input tokens (cache write): 306\n",
"100.0% of input prompt cached (187394 tokens)\n",
"Time taken: 10.73 seconds\n",
"\n",
"Turn 4:\n",
"User: What is the main theme of this novel?\n",
"Assistant: The main theme of \"Pride and Prejudice\" is the interplay between pride and prejudice in relationships, particularly as shown through the central romance between Elizabeth Bennet and Mr. Darcy. However, there are several important related themes:\n",
"\n",
"1. Pride and Prejudice:\n",
"- Darcy's pride in his social position initially makes him appear arrogant and disdainful\n",
"- Elizabeth's prejudice against Darcy is based on first impressions and incomplete information\n",
"- Both characters must overcome these flaws to find happiness together\n",
"\n",
"2. Marriage and Social Class:\n",
"- The novel explores different types of marriages (for love, security, or social advancement)\n",
"- Social class differences and their impact on relationships\n",
"- The economic necessity of marriage for women in the period\n",
"- Contrast between marriages based on true understanding (Elizabeth/Darcy, Jane/Bingley) versus those based on superficial attractions (Lydia/Wickham) or practical considerations (Charlotte/Collins)\n",
"\n",
"3. Reputation and Social Expectations:\n",
"- The importance of reputation, particularly for women\n",
"- Social manners and proper behavior\n",
"- The constraints placed on women in Regency society\n",
"\n",
"4. Personal Growth and Self-Knowledge:\n",
"- Elizabeth and Darcy's journey to better understand themselves and each other\n",
"- The importance of overcoming first impressions and preconceptions\n",
"- Learning from mistakes and misj\n",
"User input tokens: 4\n",
"Output tokens: 300\n",
"Input tokens (cache read): 187696\n",
"Input tokens (cache write): 298\n",
"100.0% of input prompt cached (187700 tokens)\n",
"Time taken: 7.07 seconds\n"
]
}
],
"outputs": [],
"source": [
"class ConversationHistory:\n",
" def __init__(self):\n",
Expand Down Expand Up @@ -507,7 +374,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "base",
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
Expand All @@ -521,7 +388,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.14"
"version": "3.12.7"
}
},
"nbformat": 4,
Expand Down