Allow parentheses to stay on their own lines

[andreyorst]

I've used Parinfer for quite long time already, and one thing that kinda annoys me at work is that Parinfer forces me to generate a lot of noise in merge requests which contain a lot of moved parentheses. I'm talking specifically about such examples:

(let [a 1
      ;; b 2
      ]
  ,,,)

This is perfectly valid Clojure code, and someone simply commented out one binding in the let block, but when we enable Parinfer, it asks if it can re-adjust the parentheses, and we get the following result:

(let [a 1]
  ;; b 2
  ,,,)

Which now treats ;; b 2 not as part of binding list and as a comment in the body. Which is not good both in the diff of the file, and to file's structure. Yes, Clojure has #_ ignore form, which can be used instead of ;; here:

(let [a 1
      #_#_b 2]
  ,,,)

The structure is preserved now, but the point is, that this is also a kind of noise in the commit. And another example is this:

(ns some-ns
  (:require [,,,
             ,,,
             ,,,
             ,,,
             ]))

A lot of programmers explicitly say that this makes easier for them to add or remove dependencies, and it is nicer for the diffs in commits too, as by adding single line at the end, you get single line changed in the diff, instead of two lines with delimiter. Parinfer also breaks this promise by forcing bracket to be on the last line.

Maybe Parinfer could be less aggressive here? For example if we do not edit current definition explicitly then Parinfer checks if invariant is preserved without moving parentheses, and leaves closing parentheses as is? Or if it is possible to make this aggressive behaviour entirely opt-in maybe it will become even more less friction environment?

I do agree that hanging parentheses are not the best looking thing, but noisy diffs are in fact more important thing to avoid IMO.

[sogaiu]

On a somewhat related note, for the case of comment forms, there are some people who use a "door-stop" (note the comma character):

(comment

  (+ 1 1)
  ;; => 2

  ,)

to keep the closing paren in place.

I've started to use commas for places where I want my code to be left alone -- it seems to work for some implementations of parinfer and IIRC there are some other editor features that seem to leave the paren alone when they might otherwise move it.

I don't claim this will work for some or all of the cases mentioned in the OP, but it seems to work for some useful situations.

I think eraserhd may have been present when the "door stop" was discussed in #parinfer on the Clojurians slack.

[andreyorst]

@sogaiu cool idea actually, but will probably only work for Clojure, as I don't know other Lisps and Schemes that feature some kind of character that is ignored by the reader, like commas in Clojure.

[sogaiu]

@andreyorst I think you're right that the comma character idea is not likely to work for other lisps. IIUC, neither Fennel nor Janet has a similar construct.

However, it turns out that other folks had been using [] and other constructs though before the comma was applied for this purpose. Perhaps there are other (possibly less convenient) options available for other lisps?

[andreyorst]

actually this has nothing to do with [] constructs. I've actually seen some Scheme code written in this style:

(define (some-foo params)
  (define (inner-foo params)
    (let ((some more)
          (bindings (foo)))
      (inner-foo body)
  ) )
  (inner-foo params)
)

This particular case will make Parinfer very angry, but given that the expression is balanced, and this is just another style of formatting the code, I think that this is a design misfeature. I don't know if something in the implementation prevents Parinfer from supporting such style by either not modifying it, since it is balanced until modification from the user, or by fully supporting this by preserving such structure on modification though.

Although Gassanenko style is not as widespread, it is still quite common in lisps to create lists like this. And in this case Parinfer will not delete leading newline, so perhaps maybe it also can leave trailing newline as well?

[eraserhd]

Regarding the examples using comments, I think that parinfer should have some special logic to keep the brackets after the comments, and this would include moving brackets down when a semicolon is inserted in the middle of a line with a paren trail. This fixes a couple issues, although I think it would be difficult to implement.

Regarding having closing parens on their own lines at the same level of indentation as the opening bracket -- but only sometimes -- would be a completely different algorithm, honestly, and I'm not even certain it can be done without losing much of parinfer's "emergent paredit" behavior.

I do think that parinfer could be modified or re-engineered to be less destructive. For example, maybe it could only modify the current top-level form, leaving the others as they are. This doesn't help for things like the R6RS library form. Maybe there's something narrower than "top-level"?

So that seems like three issues to me, one of them WONTFIX.

[andreyorst]

@eraserhd should I split the issue?

[sogaiu]

@andreyorst regarding: #92 (comment)

I think I found some characters that might work in place of a comma in Clojure and an overlapping different set in Janet.

It's not clear how well it will work in practice, but \f and \v appear to be treated as non-whitespace by at least some formatters.

IIUC, Clojure has a variety of characters that are treated as whitespace by the reader because IIUC it uses an underlying method from the JDK. Some details here if interested: sogaiu/tree-sitter-clojure#8

I don't know what the case is in other Lisps or Lisp-like languages, but I'm hoping to make use of some of this info in tooling for Janet. I have written some indentation and formatting code to treat space and tab in one way and other characters (e.g. \f or \v) differently.

The hope is to provide a light-weight method of expressing user intent regarding a location in a line beyond which (once indented) tooling should leave alone (which AFAIU is already how some tooling works). We already have "end of line" intent via things like new line characters, so perhaps we can entertain the idea of "start of user content" (per line).

As an example, one might express:

(def a
  \f      :fun)

Here I intend for \f to represent a form feed character. When indented in Janet, that wil be 2-spaces to the right (and one line down) of the first (. The spaces after the \f would be "left alone", so one would be free to add or remove spaces to position things in various ways, e.g.

(def a
  \f                            :fun)

Granted there are issues of aesthetics and usability, but I'm hoping to find out more via experimentation.

At least in Emacs it was fairly straight-forward to customize how \f or \v show up in one's editor, e.g. https://gist.github.com/sogaiu/18600747d415b3d47f89109ba965058e

IIUC you are / were a user of Emacs so I presume you are familiar with entering such characters via C-q and then C-l or C-k.

Things don't look so great in typical terminals, but perhaps it will end up being a worthwhile tradeoff...who knows.

[andreyorst]

thanks @sogaiu. Yeah, using such characters may serve as a dirty hack, but it is quite dirty, and basically adds unnecessary additional characters to the source files - something I'm very hesitant to. I'm not a big fan of // clang-format off and // clang-format on directives I often see in the C code, as it clutters the code. Invisible characters, are a bit worse in this regard, as you don't see those, and to know that something is there preventing auto-formatter (or Parinfer) to work, you need to carefully inspect the file.

I believe this issue can be solved just by not modifying any newlines in the buffer at all. Parinfer has indent mode, and by indent it should mean indent - indentation change. Modifying newlines is beyond indentation change.

So maybe as there is a forceBalance toggle in Parinfer, maybe there should be a modifyNewlines toggle for this implementation. @eraserhd ?

Basically parinfer must maintain this invariant:

(
 )

[eraserhd]

@andreyorst It's an interesting thought. I'm not really sure it's possible, as I'd have to think through a bunch of cases. If you want to go through the JSON test cases and see if any of them contradicts what you are proposing, that would be interesting to know.

[andreyorst]

@eraserhd just to illustrate what I mean:

This is not done by parinfer, but by my small package for Emacs, so here I'm only showing what I've ment by moving parentheses without modifying newlines. Note, this keeps the valid invariant, so if I enable Parinfer on this code it will not complain about indentation, only about newlines.

[andreyorst]

If you want to go through the JSON test cases and see if any of them contradicts what you are proposing, that would be interesting to know.

@eraserhd I'm not very familiar with this method of testing, but it seems to me that every test, that expects that dangling parentheses will be collapsed to single line will fail with such change. But this is an expected failure, so tests would just need to be updated. Other tests should not fail, as everything else seems to continue working as intended.

[gilch]

On Emacs Lisp, which doesn't have the #_ macro, I use .() as the doorstop. . nil also works in theory, but it's a bit longer. This should work in any Lisp dialect that supports the improper list notation (so not Clojure), unless you need to doorstop an improper list for some reason, but I've never seen that happen.

I'm vaguely remembering that some Clojure formatter or structural editing commands don't keep the , doorstop, but a #_. (full-stop doorstop) or #_/ (XML doorstop) still works. One could discard any expression, but those are the ones I've seen used.

Are there any Lisps without a workable doorstop?