Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved SemanticScholar search #96

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Conversation

r-wrobel
Copy link

@r-wrobel r-wrobel commented May 3, 2024

SemanticScholar:

  • Added improved search presented at SemanticScholar search optimization #94 (fastsearch_semanticscholar())
  • JSON import load_semanticscholar_json()
  • Two functions to get a list of all referenced docs generate_reference_list() and download the metadata mass_fetch_semanticscholar()
  • Bugfix in crossref.py
  • Temporary bugfix (?) in types.py: Skip None values when matching ids. Bug is similar to DocumentIdentifier.matches() is case-sensitive #95: Detected multiple paper, which were matched as references, because the only id beside title was None.

Wrobel added 6 commits May 3, 2024 17:02
* Add PubMed to id
* Add ability to pass extra parameters to query (request_query and request_paper). This will be needed for new functions.
* Add missing timeout to request_query
* Exclude "Too Many Requests" and "Endpont request timed out" Errors from raising an Exception and return magic-string "TIMEOUT". search_semantic will retry with increasing interval until success or other error type.
@stijnh
Copy link
Member

stijnh commented May 17, 2024

Thanks for the PR. This looks very promising! The SemanticScholar backend definitely needed some more work!

It seems to be many small improvements, I will need to go over the changes carefully so that may take some time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants