Skip to content

Latest commit

 

History

History
139 lines (102 loc) · 3.42 KB

README.md

File metadata and controls

139 lines (102 loc) · 3.42 KB

IITM_BOT Project

This project consists of an interactive chatbot (IITM Infobot) and a FastAPI backend. The chatbot helps students with questions about the IITM BS degree program by providing information on courses, the application process, and general advice.

Project Structure

  • frontend/: Contains the Chainlit-based chatbot frontend.
  • backend/: Contains the FastAPI backend and related components.

Setup

  1. Clone the Repository:

    git clone https://github.com/Eyuvaraj/LLM-Hackathon-IITM.git
cd LLM-Hackathon-IITM

  2. Environment Variables:

    • Copy and Rename .env-sample to .env in backend/ directory and add your API keys and settings: 
      HF_TOKEN=your_hf_token_here
GROQ_API_KEY=your_groq_api_key_here
NOMIC_API_KEY=your_nomic_api_key_here
base_url=local_llm_url
api_key=local_llm_api_key
dev=True

Running the Project

Backend

  1. Navigate to the Backend Directory:

    cd backend

  2. Create and Activate a Virtual Environment (optional but recommended):

    python -m venv venv
source venv/bin/activate  # On Windows use `venv\Scripts\activate`

  3. Install Backend Dependencies:

    pip install -r requirements.txt

  4. Start the FastAPI Backend:

    uvicorn api:app --reload --port 5000

Frontend

  1. Navigate to the Frontend Directory:

    cd frontend

  2. Create and Activate a Virtual Environment (optional but recommended):

    python -m venv venv
source venv/bin/activate  # On Windows use `venv\Scripts\activate`

  3. Install Frontend Dependencies:

    pip install -r requirements.txt

  4. Start the Chainlit Chatbot:

    chainlit run app.py -w

Docker Instructions

You can also use Docker to run the backend and frontend components.

Backend

  1. Navigate to the Backend Directory:

    cd backend

  2. Build Docker Image:

    docker build -t IITM-bot-backend .

  3. Run Docker Container:

    docker run -p 5000:5000 IITM-bot-backend

Frontend

  1. Navigate to the Frontend Directory:

    cd frontend

  2. Build Docker Image:

    docker build -t IITM-bot-frontend .

  3. Run Docker Container:

    docker run -p 8000:8000 IITM-bot-frontend

Usage

Once the backend and chatbot are running, you can interact with the IITM_BOT by navigating to the Chainlit interface in your web browser at http://localhost:8000/. The bot will assist you with queries related to the IITM BS degree program.

Note on API Integration

Due to limitations in handling context length in the provided API, I've opted to integrate with GROQ lamma70b instead. But meta-llama/Meta-Llama-3-8B-Instruct can be used by setting the dev variable in .env to False.

if dev:

Model: llama3-70b-8192
RAG score filter: 0.7
top_K: 10

else:

Model: Meta-Llama-3-8B-Instruct
RAG score filter: 0.5
top_K: 2

Embedding Files

The project uses the following embedding files:

  • backend/embeddings.py: Contains the code for generating embeddings and upserting embeddings into chromaDB.
  • backend/test_embeddings.py: Contains the code for testing the embeddings through the terminal.