-
-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Slow performance when opening files or invoking functions #39
Comments
We need to add caching, it would fix the lag. I was expecting somebody with a large vault to comment.
…On 5 Dec 2022 at 18:10 +0000, Nadav Spiegelman ***@***.***>, wrote:
Love the idea of this package!
I have about 5000 files and I'm seeing a delay of a few seconds whenever I open a file in my Obsidian path, or callobsidian-jump, obsidian-backlink-jump, etc. I'm using a pretty much out of box Doom config. Files within my obsidian directory open quickly when obsidian.el isn't configured.
Is this expected? Anything I can do to help debug?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>
|
I figured as much after looking through the code 😀 |
I would be very grateful for a PR :)
…On 5 Dec 2022 at 18:52 +0000, Nadav Spiegelman ***@***.***>, wrote:
I figured as much after looking through the code 😀
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: ***@***.***>
|
I'm not sure my lisp chops will allow for that, but I may try. |
I'll be glad to help. Don't have time to do it quickly myself, but I'll be glad to comment on your efforts.
…On 5 Dec 2022 at 19:20 +0000, Nadav Spiegelman ***@***.***>, wrote:
I'm not sure my lisp chops will allow for that, but I may try.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: ***@***.***>
|
Did this issue get fixed for anyone? Jumping to notes is still slow for me. |
yes |
In my own branch I've commented out the |
im also experiencing slowness
@licht1stein would be nice to have an option to disable this in the config |
Sure, I will look into it when I have time, if somebody else doesn't implement it |
I did a bunch of tests today. For my use case, I managed to make things 1000 times faster. I've made a very very low quality commit of the things I've tried here. Here's a summary:
Here's some utilities that I used over and over to help be optimize stuff: (defmacro fsta/comment (&rest body)
"Comment BODY."
nil)
(defmacro fsta/measure-time (&rest body)
"Measure the time it takes to evaluate BODY."
`(let ((time (current-time))) ;; TODO Time should be gensym'd
(prog1
(progn ,@body)
(message "%.06fs" (float-time (time-since time))))))
(defun fsta/profile (fn)
(unwind-protect
(progn
(profiler-start 'cpu+mem)
(funcall fn)
(profiler-report))
(profiler-stop))) Here are the functions that takes the most time:
Here are some other ideas that I didn't have time to test:
(I might make a PR, if I have time, no promise 😅) |
Wow, this is great work! I would be very grateful for the PR, with exception of the projectile-related part. I also think that hash-tables would be a better way to handle things. |
Notes for myself:
|
That would actually be a great addition to readme |
It would have been funny if it was already there, because I didn't even look at the readme 😆 |
I find the lag on Here is my approach (which (defvar dima-obsidian-vault-path "~/Documents/Obsidian Vault/"
"The directory to the Obsidian Vault.")
(defun dima-obsidian-update-after-save-hook ()
(when (s-starts-with-p default-directory (f-expand dima-obsidian-vault-path))
(message "Updating Obsidian Vault...")
(obsidian-update)))
(add-hook 'after-save-hook #'dima-obsidian-update-after-save-hook) Then create a custom Mine also invokes (defun dima-obsidian-jump ()
"Jump to Obsidian note.
Patches:
- remove slow call to `obsidian-update'
- when no target is there, create a new note"
(interactive)
(let ((gc-cons-threshold most-positive-fixnum))
(let* ((files (obsidian-list-all-files))
(dict (make-hash-table :test 'equal))
(_ (-map (lambda (f) (puthash (file-relative-name f obsidian-directory) f dict)) files))
(choices (-sort #'string< (-distinct (-concat (obsidian--all-aliases) (hash-table-keys dict)))))
(choice (completing-read "Jump to: " choices))
(target (obsidian--get-alias choice (gethash choice dict))))
(if target
(find-file target)
;; inline `obsidian-capture' without title variable
(let* ((filename
(s-concat obsidian-directory "/" obsidian-inbox-directory "/" choice ".md"))
(clean-filename (s-replace "//" "/" filename)))
(find-file (expand-file-name clean-filename) t)
(save-buffer)
(add-to-list 'obsidian-files-cache clean-filename)))))) Still, |
I've put together a prototype branch to experiment with trying to improve performance using some of the ideas from this thread. The main change is that the files cache is now a hashtable instead of a list, with the values being a metadata hash table holding the tags, aliases, and links.
This simple change ended up leading to many other changes as it could then made sense to use the information in the cache for many other functions. If you do a diff you'll see that I've changed a lot. It's definitely more than is necessary to change as much of this was just me renaming things, removing things, and moving things around to help me better understand how things works. This is definitely not a PR, I just made it a WIP PR to make it easier to view and see the diffs to help get feedback. This hashtable cache is populated once, and then an 'after-save-hook' is added to repopulate things for the saved file to keep things up to date. I think this should work well for files modified within I did look at emacs's I've added a few more tests to help make sure I wasn't totally breaking things, but I'd definitely like to have some more that test more of the tags and alias functionality. One things I'd like to explore more is whether it makes more sense to have a single data structure as the source of data for all functions, or whether parallel function-specific data structures should be maintained that are kept in sync with the files cache. For example, we currently use a separate aliases hashtable that is used for alias-related functions. I did implement the necessary changes so that this hashtable is updated when the files cache is updated. But this does add additional complexity when it may be find to only generate the aliases hashtable when it's needed. Similarly, we could have a tags data structure and a backlinks data structure. That would make retrieving this information, but it would certainly complicate the code with more happening on every update. A tags hashtable is used when jumping to a tag, and this is generated at the time of the call to find a tag. It's a simpler implementation at the cost of more computation happening at the time of a function call. |
I've been using that branch referenced above for the past few months and it's been working really well, so I cleaned up the code a bit and submitted it as PR #100. There are a lot of changes and I've been the only one banging on it, so I'd appreciate any feedback other may have. It's very responsive and has even allowed me to create a backlinks panel to immediately show all of the backlinks when I open a new note file, although I didn't include that in the PR as I was trying to keep it as minimal as possible. |
Love the idea of this package!
I have about 5000 files and I'm seeing a delay of a few seconds whenever I open a file in my Obsidian path, or call
obsidian-jump
,obsidian-backlink-jump
, etc. I'm using a pretty much out of box Doom config. Files within my obsidian directory open quickly when obsidian.el isn't configured.Is this expected? Anything I can do to help debug?
The text was updated successfully, but these errors were encountered: