feat: confluence updater #8
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ation type hint in delete_document method
…licator, and add corresponding default implementations
- Introduced `UploadRequest` model for handling information piece uploads. - Updated `upload_information_piece` endpoint to utilize `UploadRequest`. - Refactored `BaseRagApi` and `QdrantDatabase` to support new upload logic. - Implemented collection duplication functionality in `DefaultCollectionDuplicator`. - Enhanced `VectorDatabaseSettings` with new attributes for collection management. - Added cleanup logic for old collections in `QdrantDatabase`. - Updated tests to cover new functionality and ensure proper integration. - Removed unused dummy test files and cleaned up imports across modules.
…e loader and document uploader
…model and update vector database interface
…e and modify information piece remover and uploader to use it
…ings and initialization parameters
…r and DefaultConfluenceUpdater to utilize it
…t latest collection usage; enhance QdrantDatabase with alias management; add tests for collection duplication and mock environment variables
…ment - Updated langchain and langchain-community dependencies in pyproject.toml. - Added a dummy test in dummy4_test.py for basic functionality verification. - Created __init__.py for the endpoints test directory. - Implemented a comprehensive testing setup in conftest.py for Qdrant client and vector database. - Added tests for collection duplication and switching in default_collection_duplicator_test.py and default_collection_switcher_test.py. - Ensured proper alias management and logging during collection operations.
…ependencyContainer and VectorDatabase
…n handling in QdrantDatabase; enhance tests for collection switching
- Removed unused imports in `default_confluence_loader.py`, `default_confluence_updater.py`, and `default_document_deleter.py`. - Added missing commas in function calls for better formatting in `default_confluence_loader.py` and `default_confluence_updater.py`. - Updated type hints for function parameters in `default_document_deleter.py` and `default_confluence_updater.py`. - Enhanced formatting and comments in `confluence_parameters.py` for clarity. - Adjusted `pyproject.toml` to include additional error codes in per-file ignores. - Cleaned up unnecessary whitespace and comments in various files, including `default_collection_switcher.py` and `qdrant_database.py`. - Improved test code formatting and consistency in `conftest.py`, `default_collection_duplicator_test.py`, and `default_collection_switcher_test.py`. - Fixed minor issues in test assertions for better accuracy. - Added missing return statements in fixture functions to ensure proper behavior.
…Pydantic v2 compatibility
…liases_of_interest method for improved alias management
…tings; update tests for new functionality
… settings and mappings for consistency
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds support for a new “update” operation on Confluence spaces, enabling asynchronous extraction of Confluence content and safe switching of collections in our vector database. It introduces: