lein marg fails when a map value symbol contains a double colon

[vandr0iy]

Hi! Me and my team use some sort of a hungarian notation in our project, which goes like this: mk:stuff when it yields a data structure, mkfn:stuff when it yields a function that returns a data structure, etc.

This issue can be reproduced by creating a bunch of symbols with double colons in them, and then use them as values in a hashmap, like this

{:foo mk:foo
 :bar  mk:bar
 :baz   mk:baz}

It doesn't help if I quote them, 'mk:bar, or use only as a symbol and try to var-get it later #'mk:foo.

This is the error message I get:

Exception in thread "main" java.lang.RuntimeException: Problem parsing near line 53 <)> original reported cause is java.lang.RuntimeException: Map literal must contain an even number of forms -- java.lang.RuntimeException: Map literal must contain an even number of forms, compiling:(/tmp/form-init3307654883968985504.clj:1:73)
        at clojure.lang.Compiler.load(Compiler.java:7442)
        at clojure.lang.Compiler.loadFile(Compiler.java:7368)
        at clojure.main$load_script.invokeStatic(main.clj:277)
        at clojure.main$init_opt.invokeStatic(main.clj:279)
        at clojure.main$init_opt.invoke(main.clj:279)
        at clojure.main$initialize.invokeStatic(main.clj:310)
        at clojure.main$null_opt.invokeStatic(main.clj:344)
        at clojure.main$null_opt.invoke(main.clj:341)
        at clojure.main$main.invokeStatic(main.clj:423)
        at clojure.main$main.doInvoke(main.clj:386)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.lang.Var.applyTo(Var.java:700)
        at clojure.main.main(main.java:37)
Caused by: java.lang.RuntimeException: Problem parsing near line 53 <)> original reported cause is java.lang.RuntimeException: Map literal must contain an even number of forms -- java.lang.RuntimeException: Map literal must contain an even number of forms
        at clojure.lang.LispReader.read(LispReader.java:294)
        at clojure.lang.LispReader.read(LispReader.java:198)
        at clojure.lang.LispReader.read(LispReader.java:192)
        at marginalia.parser$parse_STAR_$fn__537$fn__540.invoke(parser.clj:160)
        at marginalia.parser$parse_STAR_$fn__537.invoke(parser.clj:159)

...

        at marginalia.parser$parse.invokeStatic(parser.clj:389)
        at marginalia.parser$parse.invoke(parser.clj:380)
        at marginalia.parser$parse_file.invokeStatic(parser.clj:418)
        at marginalia.parser$parse_file.invoke(parser.clj:415)
        at marginalia.core$path_to_doc.invokeStatic(core.clj:177)
        at marginalia.core$path_to_doc.invoke(core.clj:175)

...

        at marginalia.hiccup$eval79$fn__80.invoke(hiccup.clj:99)
        at clojure.lang.MultiFn.invoke(MultiFn.java:229)
        at clojure.lang.Var.invoke(Var.java:379)
        at marginalia.html$toc_html.invokeStatic(html.clj:198)
        at marginalia.html$toc_html.invoke(html.clj:197)
        at marginalia.html$uberdoc_html.invokeStatic(html.clj:409)
        at marginalia.html$uberdoc_html.invoke(html.clj:401)
        at marginalia.core$uberdoc_BANG_.invokeStatic(core.clj:206)
        at marginalia.core$uberdoc_BANG_.invoke(core.clj:196)
        at marginalia.core$run_marginalia.invokeStatic(core.clj:311)
        at marginalia.core$run_marginalia.doInvoke(core.clj:248)

[gdeer81]

I've literally never seen anyone name things with a colon in the middle. Thank you for the interesting case.
I'll put this on the roadmap for the 9.2 release

[gdeer81]

Further investigation shows that it can handle vars with colons; it's maps with values that have colons in them that it chokes on:
This is fine: (def trouble:map {:foo "foo"})
This is not fine: (def trouble:map2 {:my-map trouble:map})

[gdeer81]

I was able to make a minimal case for this one at the repl:
(require '[marginalia.parser :as p])
(p/parse "{:x y:z}") => Map literal must contain even number of forms

[gdeer81]

Not sure why this works at the REPL:
(def my-reader (clojure.lang.LineNumberingPushbackReader. (java.io.BufferedReader. (java.io.StringReader. (str "{:f u:y}" "\n")))))
(. clojure.lang.LispReader (read my-reader false :_eof false))

because this is exactly what parse is doing when it throws the error
see: this code
https://github.com/gdeer81/marginalia/blob/b2d82b1c84bedc4fb89430cea374143c909249c0/src/marginalia/parser.clj#L437
https://github.com/gdeer81/marginalia/blob/b2d82b1c84bedc4fb89430cea374143c909249c0/src/marginalia/parser.clj#L438

gets called by this code to make a LineNumberingPushbackReader just like I did above https://github.com/gdeer81/marginalia/blob/b2d82b1c84bedc4fb89430cea374143c909249c0/src/marginalia/parser.clj#L440

Then this code calls parse* with that reader https://github.com/gdeer81/marginalia/blob/b2d82b1c84bedc4fb89430cea374143c909249c0/src/marginalia/parser.clj#L445

so (parse "{:foo x:y}") doesn't work but
(def my-reader (clojure.lang.LineNumberingPushbackReader. (java.io.BufferedReader. (java.io.StringReader. (str "{:f u:y}" "\n")))))
(. clojure.lang.LispReader (read my-reader false :_eof false)) or (parse* my-reader) works

[camsaul]

I'm running into this too.

While not super common, I've seen symbol names with colons in them in other projects. They're fairly common in Emacs Lisp, where they're often used either for namespacing or for a group of functions that do similar things. For example in emacs-rotate there's a series of functions like rotate:main-vertical and rotate:titled that all rotate your screen layout in different ways.

We're doing something similar in Metabase where we have two functions to fetch "Cards", one to fetch Cards for a Database and another for Cards for a Table. They're named cards:database and cards:table, respectively.

[hale]

I think #163 would help, just to join the dots here.

[yogsototh]

Just a note that I experienced that issue while working on generating an XML for SAML with hiccup.
For example:

(hiccup/html
    [samlp:AuthnRequest
     {xmlns:samlp "urn:oasis:names:tc:SAML:2.0:protocol"
      :ID identifier
      :Version "2.0"
      :IssueInstant date
      :ProtocolBinding "urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST"
      :ProviderName sp-id
      :IsPassive false
      :Destination authorize-uri
      :AssertionConsumerServiceURL uri}
     [saml:Issuer
      {xmlns:saml "urn:oasis:names:tc:SAML:2.0:assertion"}
      uri]])

[tsmacdonald]

This is definitely happening because of the custom colon reader introduced in this 2011 commit that handles double-colon-namespaced keywords.

Looks like parser/read-keyword needs to be smarter, but I'm still thinking about how to do that.