org-mode integration

[fancycade]

Hi there!

Thanks for all of the effort on this project so far. This is the only emacs zettelkasten I've got to work out of the box.

One feature that I think might be missing, or maybe it is simply a matter of explaining in the docs, is how to use org-mode with neuron-mode.

The reason I bring this up is that of course emacs users tend to love org-mode, and this was one attractive quality of this project as well:
https://www.eliasstorms.net/zetteldeft/

[srid]

The bulk of the work for this needs to be done in neuron. neuron master now uses Pandoc AST, so it shouldn't be too hard. The markdown is parsed using this function:

parseMarkdown :: YAML.FromYAML meta => FilePath -> Text -> Either Text (meta, Pandoc)

Basically have a parseOrgMode that returns the Pandoc document along with the medata, and then we can update all the places in neuron to make readers polymorphic in format, and then "plug" it into neuron.

[felko]

Yeah, I'm planning on implementing org support for neuron, I already managed to get neuron generate HTML from org files (just as a quick test), the only thing left to do is to process zettel links.

On the emacs side, I'm not sure if it would require any major mode on top of org-mode, since org-mode is already configurable enough to make it pretty much as easy to use as neuron-mode, I'll try to come up with an org config that integrates neuron links using custom link types. Combined with deft, we would get all the major features of neuron-mode in org-mode (in this repo?).

[srid]

the only thing left to do is to process zettel links.

I believe regular links with the same link text as URL, like the below, will produce the pandoc AST that neuron will process as zettel links.

[[efr7d8ehf][efr7d8ehf]]

However, this might be unnecessarily verbose repetitive. In markdown one can do <efr7d8ehf>; does org-mode have something like that? Maybe internal links ...

[felko]

I think we can do [[efr7d8ehf]] and possibly use a custom link type (let's say z, which would look like [[z:efr7d8ehf]]), which allows to override behavior when following the link, and possibly specify the default description, where we could put the zettel title.

[felko]

After seeing multiple people "complaining" about neuron-mode not being available for org notes, I realize that maybe neuron-mode should be a minor mode. Most of the code is merely bridging emacs with the neuron CLI app, I think I could extract this part to a format-agnostic library, and provide minor modes for functions that require parsing links, adding tags, etc...
I'll have to do this after org support (or any other file format) is implemented in neuron.

[Whil-]

I think we can do [[efr7d8ehf]] and possibly use a custom link type (let's say z, which would look like [[z:efr7d8ehf]]), which allows to override behavior when following the link, and possibly specify the default description, where we could put the zettel title.

Ideally the ID facility of Org mode would be used for this. There are some caveats left in Org itself to support this well though. It for example relies on a global configuration file to keep track of ID's. The benefit would be that filenames and links can be decoupled, and that links will work in vanilla Org mode.

[felko]

Thanks, I was just looking at the options org mode had for this kind of link (cf. srid/neuron#263)
How would ID work? I took a quick look at how it is managed in org-brain, but I don't see the whole picture as I have never used it.

The benefit would be that filenames and links can be decoupled, and that links will work in vanilla Org mode.

There is not fundamental issue with that but neuron uses IDs for the zettels filenames. I think it would be preferable to have a uniform treatment of IDs between all note formats.

My current attempt at gluing neuron and org is this:

(defun org-follow-neuron-query (path arg)
  (neuron--follow-query
   (neuron--parse-query-from-url-or-id
    (concat "z:" path))))

(defun org-complete-neuron-link (&rest arg)
  (alist-get 'id (neuron-select-zettel)))

(defun org-activate-neuron-link (beg end path brackets)
  (let ((ov (make-overlay beg end nil t nil))
        (query (neuron--parse-query-from-url-or-id (concat "z:" path))))
    (neuron--setup-overlay-from-query ov query)))

(org-link-set-parameters "z" :follow #'org-follow-neuron-query
                         :complete #'org-complete-neuron-link
                         :activate-func #'org-activate-neuron-link)

which makes neuron links look like [[z:zettel/efr7d8ehf]] and display almost exactly like in neuron-mode. The advantage of using link types is that I can provide a :completion function to org-link-set-parameters which looks for titles. Now, I'd like to have short links, but [[z:efr7d8ehf]] doesn't work because then we can't have zettels named zettel, tags or zettels (because these would be ambiguous, e.g. [[z:tags]] is used for querying all tags). I was thinking that we could use angle links maybe, but they are poorly documented in the org-mode manual, and it looks like Pandoc's org parser doesn't recognize them, unless there's some extension that I have to enable.

Edit: Turns out Pandoc does support angle links but I haven't been able to get them to parse <efr7d8ehf>

[Whil-]

Tbh, it's probably easier to create neuron-specific links than to reuse existing link-types. My angle only is that it would be nice if org mode internals could be reused as much as possible. But the concept of query-links that Neuron has doesn't exist in Org, so for that to go into Org mode core something new would need to be developed anyways.

To write a few words about ID links since you asked: They depend on ID's being added to files or headlines within property-drawers. A node (file or headline) with such an ID can be linked to using ID-links. It uses a separate metadata file to keep track of ID locations, and with the information in that file, links work across the filesystem. But as you say, for Neuron queries and tag searches they probably won't do much good. And since ID links aren't really bound to only files they might map poorly to the design of Neuron.

To connect to thoughts about z-links, maybe there should be separate link types for zettels, for link-queries?. [[z:...]] and [[z/query:...]] for example? z-links would be as sugar around regular file-links, but limited to the Neuron context I suppose? (z-links is kind of what I hoped ID could be used for, but there are as discussed some limitations). z/query would be something else though, since that possibly could link to 0-many other zettels?

[felko]

Thanks a lot for the explanation, indeed I'm not sure ID links fit nicely into neuron's design.

But the concept of query-links that Neuron has doesn't exist in Org, so for that to go into Org mode core something new would need to be developed anyways.

I don't think that's an issue, I managed to get neuron to understand query links from org and tell org-mode to trigger neuron--follow-query when clicking a link.

I considered separating single zettel links from queries but the link syntax should be made as uniform as possible in any format.
srid suggested [[z:/efr7d8ehf]] which is a good compromise I think.

I'm also considering the possibility to let org zettels manage how they are generated in HTML using the export framework. Anyway, it'll take me some time to wrap my head around org-mode, so I'll settle for [[z:/efr7d8ehf]] links for now.

[garrett-hopper]

srid/neuron#263 has been merged and adds support for org-mode notes in Neuron.
Things like neuron--insert-zettel-link-from-id need to be updated to use the new [[z:/$id]] format when editing an org node, and the [[z://$id]] links need to be set up as org-mode hyperlink type. (See orgmode.org/manual/Adding-Hyperlink-Types.html)

[garrett-hopper]

Really rough implementation that seems to do the job

(defun org-follow-zettel-link (path)
  (org-link-open-as-file (format "%s.org" (expand-file-name (substring path 1 nil) (neuron--get-zettelkasten))) nil))

(defun org-zettel-link-active (start end path)
  (let ((o (make-overlay start end)))
    (overlay-put o 'display (format "%s" (alist-get 'zettelTitle (neuron--query-zettel-from-id (substring path 1 nil)))))))

(org-link-set-parameters
 "z"
 :follow 'org-follow-zettel-link
 :activate-func 'org-zettel-link-active)

[felko]

Thanks a lot. I still have to figure out exactly how neuron-mode as a minor mode will affect the features. I might need to release a v1.0 with the current features and work on the new version, which probably will introduce a lot of breaking change in people's configs.

[garrett-hopper]

Yeah, I'm currently not using neuron-mode at all and instead just using org-mode with some of the neuron functions.
Having neuron-mode as a minor mode would be great. :)

[beandipper]

Were you able to make any further progress? I read somewhere that you were thinking of using org-mode personally with neuron. If you are, how are you handling it now?

[felko]

No progress unfortunately, sorry. I've been busy with school recently.
Also I mostly use markdown in neuron, and occasionally org-mode when I need to execute code using org-babel or if have a really huge note that needs a bunch of nested headers (e.g. when I don't need something as organized as multiple linked zettels, but just a file in which I can throw a bunch of ideas).
Adding the features for org-mode is probably not that complicated, but turning neuron-mode into a minor-mode will involve a huge refactor presumably.