Discussion: intermediate output format

[gbaz]

This is a discussion thread. Please correct me where I'm wrong. It's just to put down a big feature idea I've been thinking of for some time in one place, and maybe one day we could move towards this.

Haddock currently has three-or-so output formats. The central format is html. There's some latex rendering, but I imagine it is quite bitrotted. There is hoogle output, which is maintained just enough to be usable, but is not very good. And finally there's an "interface" binary format used by haddock to construct hyperlinks between documentation of different packages.

What I'd like to propose a classic "separate content and presentation" refactor.

In particular, I'd suggest an output format that is slightly more abstract than html -- either xml or json. Ideally this format would be semi-fixed, or backwards compatible in its evolution. Call the files h-json files for "haddock json" or the like. The haddock executable could then be run in two modes -- one of which actually builds the source and outputs, html, h-json, whatever. The other mode need not take any source as input at all -- rather it just takes h-json files as input, and outputs the resultant html or latex or etc.

Ideally other tools besides haddock could make use of these h-json files -- ides, engines for indexing and searching haskell code, etc. Furthermore, hackage-server itself could then invoke haddock as a library to generate html on-the-fly from hjson files, and produce uniform haddocks based on the latest templates (even for packages whose h-json was created by the hackage builder years ago).

Does this seem like a plan people would be on board with?

[visortelle]

Relevant document for Rust: https://rust-lang.github.io/rfcs/2963-rustdoc-json.html

[Kleidukos]

@gbaz I agree that this would be a very good improvement.
I had postponed similar ideas in my head until @wz1000 completes Hi Haddock, but I'm fully open to discussions & proposals about the format. I would especially be interested in how we can make the task easier for Hackage, GHCi, Flora, etc.

@visortelle Thank you! Funnily enough, they cite us as prior art ;-)

[visortelle]

I'd like to implement docs pages for https://github.com/visortelle/hackage-ui

The idea that comes to my mind today is to implement Haddock output HTML to JSON parser. 🤪
It would let me generate docs pages from this JSON in any way I want.

I believe that I can do it relatively quickly. It would also allow trying docs representation JSON format before implementing it in Haddock itself.

For docs built with older versions of Haddock, show it "as is".

Of course, the idea has sense only if Haddock's HTML output is relatively stable.

@gbaz, @Kleidukos, any comments on it?

[ocramz]

Another possible format could be Pandoc (from pandoc-types : https://hackage.haskell.org/package/pandoc-types-1.22.2.1/docs/Text-Pandoc-Definition.html#t:Pandoc ).

This already comes with aeson instances; TIL pandoc already has codecs for haddock markdown, thus closing the circle.

Edit : ah sh++, I forgot Haddock is a boot package.

[bgamari]

Also relevant: haskellfoundation/tech-proposals#44