Add async support #109
Open
Add async support #109
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Implement test_read_pdf_with_url in test_file_reader.py to verify that LayoutPDFReader can successfully read and parse a PDF from a web URL - Ensures read_pdf method interacts with the actual API endpoint and returns a valid Document object
- Updated _download_pdf and _parse_pdf methods to use httpx for HTTP requests, replacing urllib3 to enable easier implementation of asynchronous operations in the future - Maintains existing functionality for PDF downloading and parsing, preserving API interactions and response handling
- Introduced async methods _download_pdf_async and _parse_pdf_async for non-blocking HTTP requests using httpx - Added read_pdf_async alongside read_pdf, supporting asynchronous local file reading with aiofiles and URL downloads - Added additional unit test for read_pdf_async (with PDF from URL) Addresses issue nlmatics#44
Open
|
Hi @ansukla, I hope this is helpful and implemented in the spirit of the project. As this is my first open-source contribution I welcome any feedback! Best, |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces asynchronous APIs and refactors existing HTTP handling to improve performance and flexibility in
LayoutPDFReader. Key changes include:read_pdf(from URL) to ensure my changes aren't breaking anything.urllib3withhttpxin_download_pdfand_parse_pdffor support of asynchronous operations._download_pdf_async,_parse_pdf_async, andread_pdf_async, allowing non-blocking operations for both HTTP requests and local file reading.This PR resolves issue #44.