From 339fcb54d8112d98e696132f3637a27659b3dfc3 Mon Sep 17 00:00:00 2001 From: Arnaud Date: Fri, 15 Sep 2023 16:01:58 +0100 Subject: [PATCH] update workshop with new titan embeddings --- 03_QuestionAnswering/01_qa_w_rag_claude.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/03_QuestionAnswering/01_qa_w_rag_claude.ipynb b/03_QuestionAnswering/01_qa_w_rag_claude.ipynb index 56874ba2..4cba18b3 100644 --- a/03_QuestionAnswering/01_qa_w_rag_claude.ipynb +++ b/03_QuestionAnswering/01_qa_w_rag_claude.ipynb @@ -234,7 +234,7 @@ "source": [ "After downloading we can load the documents with the help of [DirectoryLoader from PyPDF available under LangChain](https://python.langchain.com/en/latest/reference/modules/document_loaders.html) and splitting them into smaller chunks.\n", "\n", - "Note: The retrieved document/text should be large enough to contain enough information to answer a question; but small enough to fit into the LLM prompt. Also the embeddings model has a limit of the length of input tokens limited to 512 tokens, which roughly translates to ~2000 characters. For the sake of this use-case we are creating chunks of roughly 1000 characters with an overlap of 100 characters using [RecursiveCharacterTextSplitter](https://python.langchain.com/en/latest/modules/indexes/text_splitters/examples/recursive_text_splitter.html)." + "Note: The retrieved document/text should be large enough to contain enough information to answer a question; but small enough to fit into the LLM prompt. Also the embeddings model has a limit of the length of input tokens limited to 8192 tokens, which roughly translates to ~32,000 characters. For the sake of this use-case we are creating chunks of roughly 1000 characters with an overlap of 100 characters using [RecursiveCharacterTextSplitter](https://python.langchain.com/en/latest/modules/indexes/text_splitters/examples/recursive_text_splitter.html)." ] }, {