Welcome to the RAG (Retrieval-Augmented Generation) System repository! This project demonstrates how to implement a RAG system using Langchain in Node.js. The RAG system enhances text generation models by incorporating relevant information retrieved from external knowledge sources, such as documents or databases.
The RAG System leverages the power of Langchain to build a robust and scalable pipeline that retrieves relevant context from large documents or databases and uses this information to augment the responses generated by a language model.
- Document Parsing & Chunking: Efficiently parse large documents and split them into manageable chunks.
- Embedding & Storage: Generate embeddings for document chunks and store them in a vector database.
- Contextual Retrieval: Retrieve the most relevant chunks based on user queries.
- Text Generation: Generate responses by combining retrieved context with a language model.
- Flexible Integration: Easily integrate additional data sources and models.
- Node.js
- yarn
- OpenAI API Key (for text generation and embedding)
- Vector Database (Pinecone)
- Clone the Repository:
git clone https://github.com/varunon9/rag-langchain-nodejs.git
cd rag-langchain-nodejs
- Install Dependencies:
yarn install
- Set Up Environment Variables:
Create a .env file in the root directory and add your environment variables:
OPENAI_API_KEY=your_openai_api_key
PINECONE_API_KEY=your_pinecone_api_key
Run node .
- It will create index if not exists
- It will process sample PDF for the first time
- Processing PDF = Parsing, Chunking, Embeddings via OpenAI text-embedding-3-large model and storing embedding in Pinecone Vector db
- It will then keep accepting queries from terminal and generate answer from PDF
Check index.js for more details and to get started.