Fix/lg 11

.env.example

@@ -1,2 +1,14 @@
-OPENAI_API_KEY=your-open-api-secret-key
-USER_AGENT=Langrade/1.0
+# OpenAI Configuration
+OPENAI_API_KEY=your-api-secret-key
+OPENAI_MODEL=gpt-3.5-turbo-0125
+
+# Google (Gemini) Configuration
+GOOGLE_API_KEY=your-api-secret-key
+GEMINI_MODEL=gemini-1.5-pro-latest
+
+# Anthropic (Claude) Configuration
+ANTHROPIC_API_KEY=your-api-secret-key
+CLAUDE_MODEL=claude-3-sonnet-20240229
+
+# Default Engine Configuration
+DEFAULT_ENGINE_TYPE=openai

.github/workflows/docs.yml

@@ -0,0 +1,20 @@
+name: Publish docs via GitHub Pages
+on:
+  push:
+    branches:
+      - main
+
+jobs:
+  build:
+    name: Deploy docs
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout main
+        uses: actions/checkout@v2
+
+      - name: Deploy docs
+        uses: mhausenblas/mkdocs-deploy-gh-pages@master
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          CONFIG_FILE: mkdocs.yml
+          EXTRA_PACKAGES: build-base

.gitignore

@@ -32,3 +32,4 @@ Thumbs.db
 # Other
 create_txt.py
 file_structure_and_contents.txt
+expeiments/

README.md

@@ -1,6 +1,6 @@
 # langrade
 
-langrade is a Python library for grading and retrieving documents based on their relevance to a given question.
+langrade is a Python library for grading and retrieving documents based on their relevance to a given question. It supports multiple LLM providers including OpenAI, Anthropic (Claude), and Google (Gemini).
 
 ## Installation
 
@@ -41,11 +41,9 @@ print(f"Reasoning: {result.reasoning}")
 
 ## Features
 
-Document retrieval from web URLs
-
-Document grading based on relevance to a question
-
-Optional reasoning for grading decisions
+- Document retrieval from web URLs
+- Document grading based on relevance to a question
+- Support for multiple LLM providers (OpenAI, Anthropic, Google)
 
 ## Requirements
 

docs/advanced_usage.md

@@ -0,0 +1,60 @@
+# Advanced Usage
+
+## Custom Prompts
+
+You can customize the system prompt used by the grader:
+
+```python
+from langrade.constants import SYSTEM_PROMPT
+
+# Modify the SYSTEM_PROMPT constant
+SYSTEM_PROMPT = """
+Your custom prompt here.
+"""
+
+# Then create your grader as usual
+grader = document_grader(provider, api_key, model)
+```
+
+## Batch Processing
+
+For grading multiple documents:
+
+```python
+documents = ["doc1 content", "doc2 content", "doc3 content"]
+question = "What is AI?"
+
+results = []
+for doc in documents:
+    result = grader.grade_document(doc, question)
+    results.append(result)
+
+for i, result in enumerate(results):
+    print(f"Document {i+1}:")
+    print(f"Relevance: {result.binary_score}")
+    print(f"Reasoning: {result.reasoning}")
+    print()
+```
+
+## Custom Retriever
+
+You can create a custom retriever by subclassing BaseRetriever:
+
+```python
+from langchain.schema import BaseRetriever, Document
+
+class CustomRetriever(BaseRetriever):
+    def get_relevant_documents(self, query: str) -> List[Document]:
+        # Your custom retrieval logic here
+        pass
+
+    async def aget_relevant_documents(self, query: str) -> List[Document]:
+        # Your custom async retrieval logic here
+        pass
+
+# Use your custom retriever
+custom_retriever = CustomRetriever()
+docs = custom_retriever.get_relevant_documents("What is AI?")
+```
+
+For more advanced usage scenarios, refer to the source code and consider extending the base classes provided by Langrade.

docs/api.md

@@ -0,0 +1,63 @@
+# API Reference
+
+## document_grader
+
+```python
+def document_grader(provider: str, api_key: str, model: str = DEFAULT_MODEL) -> DocumentGrader:
+    """
+    Create a DocumentGrader instance.
+
+    Args:
+        provider (str): The LLM provider ("openai", "anthropic", or "google").
+        api_key (str): The API key for the chosen provider.
+        model (str, optional): The model to use. Defaults to DEFAULT_MODEL.
+
+    Returns:
+        DocumentGrader: An instance of the DocumentGrader class.
+    """
+```
+
+## DocumentGrader
+
+```python
+class DocumentGrader:
+    def __init__(self, provider: str, api_key: str, model: str = DEFAULT_MODEL):
+        """
+        Initialize a DocumentGrader instance.
+
+        Args:
+            provider (str): The LLM provider.
+            api_key (str): The API key for the chosen provider.
+            model (str, optional): The model to use. Defaults to DEFAULT_MODEL.
+        """
+
+    def grade_document(self, document: str, question: str) -> GradeDocuments:
+        """
+        Grade a document based on its relevance to a question.
+
+        Args:
+            document (str): The document text to grade.
+            question (str): The question to grade the document against.
+
+        Returns:
+            GradeDocuments: An object containing the grading result.
+        """
+```
+
+## create_retriever
+
+```python
+def create_retriever(urls: list[str], api_key: str) -> BaseRetriever:
+    """
+    Create a retriever for fetching relevant documents.
+
+    Args:
+        urls (list[str]): A list of URLs to fetch documents from.
+        api_key (str): The API key for the embedding model.
+
+    Returns:
+        BaseRetriever: A retriever instance.
+    """
+```
+
+For more details on the returned objects and their properties, see the source code documentation.

docs/configuration.md

@@ -0,0 +1,27 @@
+# Configuration
+
+# Provider-Specific Configuration
+
+## OpenAI
+
+```python
+from langrade import document_grader
+
+grader = document_grader("openai", "your_openai_api_key", "gpt-3.5-turbo-0125")
+```
+
+## Anthropic
+
+```python
+from langrade import document_grader
+
+grader = document_grader("anthropic", "your_anthropic_api_key", "claude-3-opus-20240229")
+```
+
+## Google
+
+```python
+from langrade import document_grader
+
+grader = document_grader("google", "your_google_api_key", "gemini-1.5-pro-latest")
+```

docs/contributing.md

@@ -0,0 +1,63 @@
+# Contributing to Langrade
+
+We welcome contributions to Langrade! Here's how you can help:
+
+## Setting Up the Development Environment
+
+1. Fork the repository
+2. Clone your fork:
+
+```bash
+git clone https://github.com/yourusername/langrade.git
+cd langrade
+```
+
+3. Set up a virtual environment:
+
+```bash
+python -m venv venv
+source venv/bin/activate  # On Windows, use `venv\Scripts\activate`
+```
+
+4. Install dependencies:
+
+```bash
+pip install -e ".[dev]"
+```
+
+## Running Tests
+
+Run tests using pytest:
+
+```bash
+poetry run test
+```
+
+## Coding Standards
+
+We use Black for code formatting and flake8 for linting:
+
+```bash
+black .
+flake8
+```
+
+## Submitting Changes
+
+Create a new branch:
+
+`git checkout -b your-branch-name`
+
+Make your changes and commit them:
+
+`git commit -m "Your detailed commit message"`
+
+Push to your fork:
+
+`git push origin your-branch-name`
+
+Submit a pull request
+
+Please ensure your code adheres to our coding standards and is well-documented.
+
+Thank you for contributing to Langrade!

docs/faq.md

@@ -0,0 +1,51 @@
+# Frequently Asked Questions
+
+## General
+
+### Q: What is Langrade?
+
+A: Langrade is a Python library for grading and retrieving documents based on their relevance to a given question. It supports multiple LLM providers including OpenAI, Anthropic (Claude), and Google (Gemini).
+
+### Q: Which Python versions are supported?
+
+A: Langrade supports Python 3.9 and above.
+
+## Usage
+
+### Q: How do I choose between different LLM providers?
+
+A: You can specify the provider when creating a `DocumentGrader`:
+
+```python
+from langrade import document_grader
+
+# For OpenAI
+grader_openai = document_grader("openai", "your_openai_api_key", "gpt-3.5-turbo-0125")
+
+# For Anthropic
+grader_anthropic = document_grader("anthropic", "your_anthropic_api_key", "claude-3-opus-20240229")
+
+# For Google
+grader_google = document_grader("google", "your_google_api_key", "gemini-1.0-pro")
+```
+
+### Q: Can I use my own custom models?
+
+A: Currently, Langrade supports the models provided by OpenAI, Anthropic, and Google. Custom model support may be added in future versions.
+
+## Troubleshooting
+
+### Q: I'm getting an "API key not found" error. What should I do?
+
+A: Ensure that you've set up your .env file correctly with the appropriate API keys. Also, make sure you're loading the environment variables in your code:
+
+```python
+from dotenv import load_dotenv
+load_dotenv()
+```
+
+### Q: How can I report a bug or request a feature?
+
+A: Please open an issue on our GitHub repository. Provide as much detail as possible, including your Python version and any error messages you're seeing.
+
+For more detailed information, please refer to our [documentation](https://langrade.readthedocs.io/).

docs/index.md

@@ -0,0 +1,42 @@
+# Welcome to Langrade
+
+Langrade is a Python library for grading and retrieving documents based on their relevance to a given question. It supports multiple LLM providers including OpenAI, Anthropic (Claude), and Google (Gemini).
+
+## Quick Start
+
+```python
+from langrade import document_grader, create_retriever
+
+provider = "openai"
+api_key = "your_api_key_here"
+model = "gpt-3.5-turbo-0125"
+
+grader = document_grader(provider, api_key, model)
+
+urls = [
+    "https://example.com/article1",
+    "https://example.com/article2",
+]
+retriever = create_retriever(urls, api_key)
+
+question = "What is AI?"
+docs = retriever.get_relevant_documents(question)
+doc_txt = docs[0].page_content
+
+result = grader.grade_document(doc_txt, question)
+print(f"Relevance: {result.binary_score}")
+print(f"Reasoning: {result.reasoning}")
+```
+
+## Table of Contents
+
+- Installation
+- Usage
+- API Reference
+- Configuration
+- Providers
+- Advanced Usage
+- Troubleshooting
+- Contributing
+- Changelog
+- FAQ