Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Working update #16

Open
wants to merge 79 commits into
base: master
Choose a base branch
from
Open

Working update #16

wants to merge 79 commits into from

Conversation

holycrepe
Copy link
Collaborator

Summary of changes:

  1. Note validation (via external script)
  2. Find deleted notes
  3. Updated see also processing
  4. Confirmed some old references to deleted functions were removed
  5. README updated

Avi Puchalapalli and others added 17 commits September 15, 2015 08:33
Added DTD Validation to makeNote/updateNote functions. These are for …
b407b20 
…uploading content to Evernote's servers
1. Move lxml code out of addon and into separate command-line script …
54bb4c9 
…since unable to bundle lxml

2. Integrated command line utility with Anknotes' Anki menu
Minor changes to logging
6aa7788
Added Command Line Tools such as Find Deleted Notes
7d1d446
Finalized Note validation and Note maintenance ('Find deleted notes')
7c59412
Class-based processing of See Also notes. This is to detect changes t…
87b3207 
…hat need to be uploaded to Evernote
Confirmed no references to generateTocTitle, all are to EvernoteNoteT…
aa6933d 
…itle.generateTOCTitle
Squashed some bugs
23dd99f
Long-overdue update for README, moved README to root, and added my /e…
c438412 
…xtra/dev folder to the repo
Merge branch 'master' into holycrepe
515424e 
Conflicts:
	.gitignore
	anknotes/Anki.py
	anknotes/AnkiNotePrototype.py
	anknotes/Controller.py
	anknotes/EvernoteImporter.py
	anknotes/EvernoteNoteFetcher.py
	anknotes/EvernoteNotePrototype.py
	anknotes/EvernoteNoteTitle.py
	anknotes/EvernoteNotes.py
	anknotes/__main__.py
	anknotes/ankEvernote.py
	anknotes/bare.py
	anknotes/constants.py
	anknotes/db.py
	anknotes/error.py
	anknotes/extra/ancillary/enml2.dtd
	anknotes/extra/ancillary/regex-see_also.txt
	anknotes/extra/ancillary/regex.txt
	anknotes/find_deleted_notes.py
	anknotes/graphics.py
	anknotes/html.py
	anknotes/logging.py
	anknotes/menu.py
	anknotes/settings.py
	anknotes/shared.py
	anknotes/structs.py
	anknotes/toc.py
Merge branch 'holycrepe'
821ecc6
Re-merge changes
91294ac
Fix api check on first load; Exclude developer automation files from …
d2d8740 
…repo
Fix api check on first load; Exclude developer automation files from …
edc851a 
…repo
Merge branch 'master' into holycrepe
7b4dcb9
Merge branch 'master' into brumar/holycrepe
86f6f05
@Scriptkiddi
fixed some startup problems
410e273
@Scriptkiddi
Copy link
Collaborator

Hey still getting import errors i could fix the most of them but there
are a few NoneType and so on

On 23.09.2015 08:11, Avi Puchalapalli wrote:

Summary of changes: 1. Note validation (via external script) 2.
Find deleted notes 3. Updated see also processing 4. Confirmed some
old references to deleted functions were removed 5. README updated

You can view, comment on, or merge this pull request online at:

#16

Commit Summary

  • Added DTD Validation to makeNote/updateNote functions. These are
    for uploading content to Evernote's servers * 1. Move lxml code out
    of addon and into separate command-line script since unable to
    bundle lxml * Minor changes to logging * Added Command Line Tools
    such as Find Deleted Notes * Finalized Note validation and Note
    maintenance ('Find deleted notes') * Class-based processing of See
    Also notes. This is to detect changes that need to be uploaded to
    Evernote * Confirmed no references to generateTocTitle, all are to
    EvernoteNoteTitle.generateTOCTitle * Squashed some bugs *
    Long-overdue update for README, moved README to root, and added my
    /extra/dev folder to the repo

File Changes

Patch Links:

— Reply to this email directly or view it on GitHub
#16.

@holycrepe
Copy link
Collaborator Author

Okay, thanks for looking into this. I'm running a code inspection with PyCharm now, and hopefully that will catch everything

Avi Puchalapalli added 4 commits September 23, 2015 09:37
Merge remote-tracking branch 'origin/holycrepe' into holycrepe
9eaed47
Rebuild template after obtaining Evernote uid, shard
d83a7bc
The Anki/Evernote imports actually need to be conditional, because th…
6079aaa 
…is file is also used by the external command line utility to validate notes. Since that utility doesn't operate in the Anki environment, trying to import these modules will result in an import error. I changed the conditional to import if it's in Anki environment.
Various Code Inspection fixes
da89852
@holycrepe
Copy link
Collaborator Author

@Scriptkiddi Unfortunately PyCharm couldn't find anything that would cause an import error. Could you screenshot the errors and paste them here?

@Scriptkiddi
Copy link
Collaborator

An error occurred in an add-on.
Please post on the add-on forum:
https://anki.tenderapp.com/discussions/add-ons

Traceback (most recent call last):
File "/usr/share/anki/aqt/main.py", line 165, in onOpenProfile
self.loadProfile()
File "/usr/share/anki/aqt/main.py", line 244, in loadProfile
runHook("profileLoaded")
File "/usr/share/anki/anki/hooks.py", line 26, in runHook
func(*args)
File "/home/fritz/Anki/addons/anknotes/main.py", line 83, in anknotes_profile_loaded
menu.upload_validated_notes(True)
File "/home/fritz/Anki/addons/anknotes/menu.py", line 140, in upload_validated_notes
controller = Controller()
File "/home/fritz/Anki/addons/anknotes/Controller.py", line 50, in init
self.evernote = Evernote()
File "/home/fritz/Anki/addons/anknotes/ankEvernote.py", line 87, in init
self.client = EvernoteClient(token=auth_token, sandbox=ANKNOTES.EVERNOTE_IS_SANDBOXED)
NameError: global name 'EvernoteClient' is not defined

@Scriptkiddi
Copy link
Collaborator

i think i fixed this by moving the evernote client import out of your if statment

@holycrepe
Copy link
Collaborator Author

Could you try this: http://pastebin.com/8111hStH

I added an assert statement at line 19. I also changed the conditional to test if Anki module is importable instead of lxml. It's possible that lxml is importable in your Anki config

@holycrepe
Copy link
Collaborator Author

Unfortunately having it outside of the conditional will break the script I'm using to run note validation. I couldn't get lxml imported into my Anki config, so I made a script that runs externally via command line as opposed to within the Anki environment as an addon

@holycrepe
Copy link
Collaborator Author

I fixed the issue with templates that was probably causing a string formatting error when you first loaded Anknotes. After your first import, Anknotes will now detect the uid and shard, and after next launch, will rebuild the templates. This is required for the template to be able to generate Evernote Note links from guids

@holycrepe
Copy link
Collaborator Author

@Scriptkiddi, Actually your Windows 10 error seems really odd to me. It looks like one of your files is out of date since:

  1. The self.anki.add_evernote_model call has been moved to Controller.py
  2. The constant GUID_FIELD_NAME has been renamed to FIELDS.EVERNOTE_GUID
  3. A search of current source code yields no results for 'source_ord' or 'source_field'

Since the self.anki.add_evernote_model call is in __main__.py for you, and the GUID_FIELD_NAME variable resolves, I think your __main__.py and constants.py are out of date.

If you have symlinks or something like that set up, maybe they are pointing to an old copy of the code. Can you confirm that they are up to date by opening anknotes\__main__.py in a text editor? Can you try re-downloading or re-cloning the holycrepe branch in this repository?

@Scriptkiddi
Copy link
Collaborator

my bad i downloaded the wrong branch i will test it later today again

@Scriptkiddi
Copy link
Collaborator

An error occurred in an add-on.
Please post on the add-on forum:
https://anki.tenderapp.com/discussions/add-ons

Traceback (most recent call last):
File "C:\Users\Scriptkiddi\Documents\Anki\addons\anknotes\menu.py", line 72, in
def auto_reload_wrapper(function): return lambda: auto_reload_modules(function)
File "C:\Users\Scriptkiddi\Documents\Anki\addons\anknotes\menu.py", line 89, in auto_reload_modules
function()
File "C:\Users\Scriptkiddi\Documents\Anki\addons\anknotes\menu.py", line 156, in import_from_evernote
controller.proceed()
File "C:\Users\Scriptkiddi\Documents\Anki\addons\anknotes\Controller.py", line 210, in proceed
self.evernoteImporter.proceed(auto_paging)
File "C:\Users\Scriptkiddi\Documents\Anki\addons\anknotes\EvernoteImporter.py", line 176, in proceed
self.proceed_import_notes()
File "C:\Users\Scriptkiddi\Documents\Anki\addons\anknotes\EvernoteImporter.py", line 235, in proceed_import_notes
self.ImportProgress.processResults(self.import_into_anki(self.ImportProgress.GUIDs.Server.New))
File "C:\Users\Scriptkiddi\Documents\Anki\addons\anknotes\EvernoteImporter.py", line 132, in import_into_anki
Results = self.evernote.create_evernote_notes(evernote_guids)
File "C:\Users\Scriptkiddi\Documents\Anki\addons\anknotes\ankEvernote.py", line 370, in create_evernote_notes
self.check_ancillary_data_up_to_date()
File "C:\Users\Scriptkiddi\Documents\Anki\addons\anknotes\ankEvernote.py", line 397, in check_ancillary_data_up_to_date
"Tags were not up to date when checking ancillary data")
File "C:\Users\Scriptkiddi\Documents\Anki\addons\anknotes\ankEvernote.py", line 537, in update_tags_database
db.executemany(enTag.sqlUpdateQuery(), data)
File "C:\Users\Scriptkiddi\Documents\Anki\addons\anknotes\db.py", line 441, in executemany
self._db.executemany(sql, data)
sqlite3.OperationalError: no such table: anknotes_evernote_tags

@holycrepe
Copy link
Collaborator Author

Okay looks like the tags table was not created on launch. I will fix that asap

@holycrepe
Copy link
Collaborator Author

Sorry for the delay... Bug is fixed with the latest commit. Please test the latest commit, and if you still get the database errors, send me a zip of your /extra/logs folder. Thanks for your efforts

@Scriptkiddi
Copy link
Collaborator

hey so i had some time to look at it a little bit longer i fixed to import errors but my time ran out on
An error occurred in an add-on.
Please post on the add-on forum:
https://anki.tenderapp.com/discussions/add-ons

Traceback (most recent call last):
File "/usr/share/anki/aqt/addons.py", line 41, in loadAddons
import(file.replace(".py", ""))
File "/home/fritz/Anki/addons/anknotes_start.py", line 6, in
from anknotes import main
File "/home/fritz/Anki/addons/anknotes/main.py", line 17, in
from anknotes.shared import *
File "/home/fritz/Anki/addons/anknotes/shared.py", line 14, in
from anknotes.constants import *
File "/home/fritz/Anki/addons/anknotes/constants.py", line 4, in
from anknotes.constants_settings import *
File "/home/fritz/Anki/addons/anknotes/constants_settings.py", line 12, in
SETTINGS = DictSettings(key='anknotes')
File "/home/fritz/Anki/addons/anknotes/dicts.py", line 92, in init
super(cls.mro()[mro], self).init(mro+1, _a, *_kw)
File "/home/fritz/Anki/addons/anknotes/dicts.py", line 79, in init
super(cls_mro, self).init(mro+1, _a, *_kw)
File "/home/fritz/Anki/addons/anknotes/dicts.py", line 11, in init
super(cls.mro()[mro], self).init(mro+1, _a, *_kw)
File "/home/fritz/Anki/addons/anknotes/dicts_base.py", line 123, in init
super(cls.mro()[mro], self).init(mro+1, _a, *_kw)
File "/home/fritz/Anki/addons/anknotes/addict/addict.py", line 46, in init
self.update(_a, *_kw)
File "/home/fritz/Anki/addons/anknotes/addict/addict.py", line 366, in update
"{0} types".format(arg.class))
TypeError: Dict does not understand <type 'int'> types
Traceback (most recent call last):
File "/usr/share/anki/aqt/addons.py", line 41, in loadAddons
import(file.replace(".py", ""))
File "/home/fritz/Anki/addons/anknotes_start_note_validation.py", line 2, in
from anknotes import stopwatch
File "/home/fritz/Anki/addons/anknotes/stopwatch/init.py", line 12, in
from anknotes.constants import ANKNOTES
File "/home/fritz/Anki/addons/anknotes/constants.py", line 4, in
from anknotes.constants_settings import *
File "/home/fritz/Anki/addons/anknotes/constants_settings.py", line 12, in
SETTINGS = DictSettings(key='anknotes')
File "/home/fritz/Anki/addons/anknotes/dicts.py", line 92, in init
super(cls.mro()[mro], self).init(mro+1, _a, *_kw)
File "/home/fritz/Anki/addons/anknotes/dicts.py", line 79, in init
super(cls_mro, self).init(mro+1, _a, *_kw)
File "/home/fritz/Anki/addons/anknotes/dicts.py", line 11, in init
super(cls.mro()[mro], self).init(mro+1, _a, *_kw)
File "/home/fritz/Anki/addons/anknotes/dicts_base.py", line 123, in init
super(cls.mro()[mro], self).init(mro+1, _a, *_kw)
File "/home/fritz/Anki/addons/anknotes/addict/addict.py", line 46, in init
self.update(_a, *_kw)
File "/home/fritz/Anki/addons/anknotes/addict/addict.py", line 366, in update
"{0} types".format(arg.class))
TypeError: Dict does not understand <type 'int'> types

since the error logs did not log anything new im not attaching them

@Scriptkiddi
Copy link
Collaborator

oh also please keep all external libaries in the anknotes folder since its a really bad practice installing the own deps into folders where they dont belong

@holycrepe
Copy link
Collaborator Author

Thanks for looking into it! Now that I have some experience with Python (I never coded in Python before this project), I am in the process of trying to remove all the crappy code that has made it into my branch in the process of trying to learn the language. So in the next few days I should have a build without most of the problems with the current branch (For example, using a nested dictionary for the settings object, while it saved time when creating new settings, seems like unnecessary overhead).

Do you mean to put the dependencies in the root folder of the repo (along with anknotes_start.py) or the /anknotes folder along with __main__.py?

@Scriptkiddi
Copy link
Collaborator

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

I mean putting the bs4 lib in the anknotes folder you can not move the
anknotes_start in the anknotes folder because then anki would not load
the plugin

On 16.11.2015 23:22, Avi Puchalapalli wrote:

Thanks for looking into it! Now that I have some experience with
Python (I never coded in Python before this project), I am in the
process of trying to remove all the crappy code that has made it
into my branch in the process of trying to learn the language. So
in the next few days I should have a build without most of the
problems with the current branch (For example, using a nested
dictionary for the settings object, while it saved time when
creating new settings, seems like unnecessary overhead).

Do you mean to put the dependencies in the root folder of the repo
(along with |anknotes_start.py|) or the |/anknotes| folder along
with |main.py|?

— Reply to this email directly or view it on GitHub
#16 (comment).

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJWSlddAAoJEMlXFywT0VZDNqYH/RHT0TytASDKb8OVqoz9+FZ9
yLYI48n3uPyC434quw6v+sq+UnoHL+5ChhJ3KCMU1NgurtGVcD4TmWtwROoMLeF5
yGoSh5imauCEORnsRVimriGSdrCfqq9zXw7PY0lRmDI9CBvka4uijYELhTVGuT5L
EKVCQljuYO4X0TkvxR2wGN1XyDMVx4VJHgtd1qSUOCNGFmV5u4UqFMKCmrmwCwaD
F2+smduov8bN+ZfpO1HMusPhnE1Eh/PSUGmBy1n+HX7S+DZ437pzCClZ9IjXEtdy
HoXKNPVxLrf1D+gsnqPNEFndAN5v4jiAo0hItyc6mcE7FnSt6vmujh7KlRCkvYc=
=G9hY
-----END PGP SIGNATURE-----

@holycrepe
Copy link
Collaborator Author

Oh okay so move bs4 from the repository root to the same folder as anknotes/__main__.py? The only issue with that is that the imports in bs4 and other libraries all assume that it is a top-level package; So I would have to prefix all the imports in bs4 with 'anknotes.' so that it is properly routed to the anknotes folder. The other option would be converting bs4's imports from absolute to relative

This is fairly simple for bs4 since it only has a few import statements, but I can see how it can get complicated. I see that the evernote, oauth2, and thrift libraries' imports are all prefixed with anknotes.. Did that have to be done manually or is there a tool that will do that for us?

I already updated the imports for bs4 manually as you recommended, but I'm just curious for future reference

@Scriptkiddi
Copy link
Collaborator

i did it manually but im pretty sure there are tools for that kind of stuff

@sjobeek
Copy link

sjobeek commented Apr 8, 2016

Hey guys, this is some interesting work.

I'm a python dev and looking to potentially use this for my own study and of course contribute back fixes I may need to make.

Which branch would you suggest I start with, and do you have any tips or obvious pitfalls I should avoid when trying to get this up and running on my system for the first time?

@Scriptkiddi
Copy link
Collaborator

Hey I would suggest you either start from holycrepes branch but there I dont know how far he is, I have not touched the project in a while or you take the master branch which is far behind but relaltivly stable. I would recommend skimming through the evernote api guide lines and geoi tting familiar how anki works internally (decks, notes, cards).

@holycrepe
Copy link
Collaborator Author

@sjobeek My branch has some improvements but I'm not sure about its stability or whether I have pushed all local changes. I have an important exam coming up next week, but I'll be happy to look into this for you the week after. I updated the readme on my repository so that should give you a good start.

In the meantime, follow @Scriptkiddi's suggestions. If you can try to reproduce the bug @Scriptkiddi mentioned above, that would be very helpful

@brumar
Copy link
Owner

brumar commented Apr 8, 2016

Hello and welcome. I am bit out of sync with this project, I can't be very helpful at the moment. As a matter of practical use of the plug-in, I tried to find you the last version of the plugin I uploaded on anki/addons. But I can't find it anymore. Very probably because my account has expired...

@KrishaanKhubchand
Copy link

hey guys, I've been trying to install this - But to no avail, any suggestions?

@holycrepe
Copy link
Collaborator Author

Are you having issues with this PR or brumar/master?

@brumar
Copy link
Owner

brumar commented Nov 15, 2017

Hi Holycrepe !
I hope you are doing well.
I fixed the problem on my repo and added a token protection feature. @KrishaanKhubchand and I talked by private message and now it seems to work for him. I also re-upload the addon here https://ankiweb.net/shared/info/999519295 . I also tried again to test your fork but with no success :(. By the way, are you the uploader of this one https://ankiweb.net/shared/info/1619377479 ? This is just a zip version of your repository, so that can't possibly work.
Cheers !

@holycrepe
Copy link
Collaborator Author

Hey Bruno, my AnkiWeb account expired 04/2016 so the upload is not mine. I'm glad you've been able to revive the project. I've had to reformat since working on anknotes, so I don't have a way to work on development right now. If I can restore the repo from a backup, I may give it a shot.

Is there a specific issue with the fork? I was unable to reproduce any errors, and the fork was working as intended last time I used it, although it has been a long time.

@brumar
Copy link
Owner

brumar commented Nov 15, 2017

I got the same problem of account expiration which discouraged me a bit, because at that time I made a nice user documentation on the addon page (but without backup, shame on me). If you want to work again on this project I can retry to see what kind of specific error I got.

@holycrepe
Copy link
Collaborator Author

Yes please, I would appreciate that. Guess we'll have to add the addon description to the repo so it's under version control! I have some exams coming up and it will be helpful to get my fork working. I'll be reinstalling anki, so maybe I'll be able to reproduce the error with a fresh environment and machine

@brumar
Copy link
Owner

brumar commented Nov 15, 2017 •
edited
Loading

Guess we'll have to add the addon description to the repo so it's under version control!

My thoughts exactly ahah : https://github.com/brumar/anknotes/blob/master/addonDescription.html

@holycrepe
Copy link
Collaborator Author

Ah nice, I've committed a quick fix to that description: 1ab3fed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@holycrepe @Scriptkiddi @brumar @sjobeek @KrishaanKhubchand