Skip to content
This repository has been archived by the owner on Jan 31, 2023. It is now read-only.

Improved dataframe handling and save declined matches also #20

Open
wants to merge 92 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
92 commits
Select commit Hold shift + click to select a range
2f4d471
Rewrite with new CrossrefSubject class that handles all the subject s…
dpriskorn Feb 1, 2022
303a54a
Fix logging variable
dpriskorn Feb 1, 2022
4efeedc
match_cache.py: Guard None
dpriskorn Feb 1, 2022
79b9b1a
subject.py: Add missing attribute
dpriskorn Feb 1, 2022
cad059a
ontology.py: Guard None
dpriskorn Feb 1, 2022
27c49d0
ontology.py: Fix not bug
dpriskorn Feb 1, 2022
9f8c35e
subject.py: Add missing assignment when declined
dpriskorn Feb 1, 2022
7661ef4
subject.py: Fix bug with f-string
dpriskorn Feb 1, 2022
e967143
doi.py: lookup_and_match_subjects(): Only match subjects if found in …
dpriskorn Feb 1, 2022
d11b22e
doi.py: Guard for doi being empty string
dpriskorn Feb 1, 2022
5a0ea95
scientific_item.py: Guard for doi being empty string
dpriskorn Feb 1, 2022
b4f62ae
Many changes. Fix bug with uploads not working.
dpriskorn Feb 1, 2022
eaed7ef
Update requests to minimum requirements by other packages
dpriskorn Feb 2, 2022
f0c5774
requirements.txt: Add case-converter
dpriskorn Feb 2, 2022
c008a0e
config.example.py: Add statistic_pickle_filename
dpriskorn Feb 2, 2022
78eee67
ontology.pkl: New pickle file with the domain specific science ontolo…
dpriskorn Feb 2, 2022
ab5bfe6
ontology.pkl: Switch to protocol 4 to support older python versions.
dpriskorn Feb 2, 2022
9e64353
Rename config variable
dpriskorn Feb 2, 2022
e9acb6c
Handle DOI being empty string better
dpriskorn Feb 2, 2022
9760bdf
engine.py: Catch also JSONDecodeError from habanero
dpriskorn Feb 2, 2022
bad70df
wikipedia_page.py: Remove extra press enter to continue that is not n…
dpriskorn Feb 2, 2022
52e9c93
Show all matches before upload instead
dpriskorn Feb 2, 2022
820ac8f
Move table functions to own file to avoid import errors.
dpriskorn Feb 2, 2022
8308b12
Fix import errors.
dpriskorn Feb 2, 2022
5ad13eb
Fix list comprehension
dpriskorn Feb 2, 2022
524c325
Improve message of input dialog and show only once before uploading.
dpriskorn Feb 2, 2022
27bdb66
Test the DOI from Wikidata before working on it.
dpriskorn Feb 2, 2022
ac97e1c
Move debug printing of match.dict() and cache.dict() to before the qu…
dpriskorn Feb 2, 2022
806577e
Lower message from warning to debug.
dpriskorn Feb 2, 2022
2055f7e
Improve question message with color
dpriskorn Feb 2, 2022
45329b4
Bold red for debugging the location from hub when not a valid QID
dpriskorn Feb 2, 2022
b459255
Lower message from warning to debug.
dpriskorn Feb 2, 2022
f497fac
Enable debug of work model
dpriskorn Feb 2, 2022
ae37be0
Make work title green and bold
dpriskorn Feb 2, 2022
83a1604
Move message into table
dpriskorn Feb 2, 2022
c10e27f
Fix imports
dpriskorn Feb 2, 2022
291d1ce
tables.py: Guard for ner being None.
dpriskorn Feb 2, 2022
16d6c8b
Rename attribute ner -> named_entity_recognition
dpriskorn Feb 2, 2022
4fdd8f7
Remove old comment
dpriskorn Feb 2, 2022
a7098e7
Rename a lot of classes and files to improve the readability, move en…
dpriskorn Feb 2, 2022
1360fe7
Fix validation issues
dpriskorn Feb 2, 2022
d752444
Fix Doi import errors
dpriskorn Feb 2, 2022
0065b77
Add remaning icons to classes.puml
dpriskorn Feb 2, 2022
8f032a8
Fix missing import
dpriskorn Feb 3, 2022
219d28c
doi.py: New public method lookup_in_wikidata() and use it
dpriskorn Feb 3, 2022
7fc55d4
Fix pickle bug and rename attribute to __pickle_filename
dpriskorn Feb 3, 2022
357b449
Switch from value error to WARNING when DOI is empty string
dpriskorn Feb 3, 2022
6a7d017
Improve pickle handling and remove dead code
dpriskorn Feb 3, 2022
0490049
Fix bug with _pickle_filename and add more debug input.
dpriskorn Feb 3, 2022
c045b6d
Untangle the circular mess of Doi and ScientificItem containing each …
dpriskorn Feb 3, 2022
7eea212
New CrossrefEntryType value
dpriskorn Feb 3, 2022
8709a40
wikipedia_page.py: Improve message
dpriskorn Feb 3, 2022
bc02bbe
Divide __init__() into smaller methods
dpriskorn Feb 3, 2022
c193ec0
Move each crossref class to it's own file and sort the attributes.
dpriskorn Feb 3, 2022
4f4792a
classes.puml: Group in packages for better overview.
dpriskorn Feb 3, 2022
c979d9c
classes.puml: Add links between classes
dpriskorn Feb 3, 2022
eccb262
Fix syntax
dpriskorn Feb 3, 2022
3059166
Try make variable public to fix pydantic error
dpriskorn Feb 3, 2022
bf26592
Enable arbitrary types
dpriskorn Feb 3, 2022
41ee461
wikipedia_page.py: __upload_all_subjects_matched_to_wikidata__(): Upd…
dpriskorn Feb 3, 2022
644677e
scientific_item.py: Remove confusing INFO message.
dpriskorn Feb 4, 2022
17a3dd2
scientific_item.py: lookup_and_match_subjects(): Add missing lookup i…
dpriskorn Feb 4, 2022
658954e
tables.py: print_all_matches_table(): Fix call to Doi attribute
dpriskorn Feb 4, 2022
2f93802
Move input to only fire on DEBUG
dpriskorn Feb 4, 2022
4784c1d
Fix call to subject_matches and switch message to INFO
dpriskorn Feb 4, 2022
7676769
Handle buggy response from Hub
dpriskorn Feb 4, 2022
00c1139
Rename config variables and add new constant wikidata_wiki_prefix an…
dpriskorn Feb 4, 2022
3be7f3a
Silence common Resource not found error from Crossref
dpriskorn Feb 4, 2022
84a6724
New type report in Crossref
dpriskorn Feb 4, 2022
75fbb76
Fix Resource not found handling.
dpriskorn Feb 4, 2022
547c44d
Move red debug message to DEBUG
dpriskorn Feb 4, 2022
85e265a
Big refactor of classes to simplify the structure of the project
dpriskorn Feb 4, 2022
b0020ed
Update README.md
dpriskorn Feb 4, 2022
c56da0f
scientific_item.py: Fix bug with Hub url
dpriskorn Feb 4, 2022
7b18fd7
Fix import
dpriskorn Feb 5, 2022
a54c759
Fix import
dpriskorn Feb 5, 2022
ba089a6
Guard None
dpriskorn Feb 5, 2022
c0a80be
Guard None
dpriskorn Feb 5, 2022
cf9c962
Fix message
dpriskorn Feb 5, 2022
e44785a
Fix more logging in work.py
dpriskorn Feb 5, 2022
ddc5653
Fix raw_subjects bug.
dpriskorn Feb 5, 2022
f946551
Guard None
dpriskorn Feb 5, 2022
a0668ba
work.py: Guard for number_of_matches being None
dpriskorn Feb 5, 2022
7bbb748
Support posted-content
dpriskorn Feb 5, 2022
c57a00d
work.py: We no longer care about the object_type because we have no u…
dpriskorn Feb 5, 2022
3d019eb
Remove comment
dpriskorn Feb 6, 2022
acc8b5d
__init__.py: No longer exclude books or any other types.
dpriskorn Feb 7, 2022
58f2a9f
work.py: Remove old import
dpriskorn Feb 7, 2022
364b870
Try renaming private variable to __x__
dpriskorn Feb 7, 2022
ccb5339
Add mypy configuration
dpriskorn Feb 9, 2022
245bece
Ignore typing for 3 imports
dpriskorn Feb 9, 2022
6a36177
Implement Typed Argument Parser and mypy and fix all warnings.
dpriskorn Feb 9, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
scientific_item.py: Guard for doi being empty string
dpriskorn committed Feb 1, 2022

Verified

This commit was signed with the committer’s verified signature.
antonioribeiro Antonio Ribeiro
commit 5a0ea956a00d35bfca37ad565d2756c36e4f8d55
2 changes: 0 additions & 2 deletions asseeibot/models/wikimedia/wikidata/scientific_item.py
Original file line number Diff line number Diff line change
@@ -144,8 +144,6 @@ def __lookup_via_hub__(self) -> None:
raise ValueError("doi was empty string")
logger.info("Using DOI from Crossref to lookup in Hub")
self.__call_the_hub_api__(doi)
if self.doi == "":
raise ValueError("doi was empty string")
if not self.found_in_wikidata:
logger.info("Using DOI from Wikipedia to lookup in Hub")
self.__call_the_hub_api__(self.doi.value)