Index pages

[EmileTrotignon]

This PR improve driver generated pages for indexes of packages, by adding the list of libraires and their modules.

[EmileTrotignon]

[dbuenzli]

We agree your example should be using brr's index.mld page, instead of this, right ?

For automatically generated pages, I think it would be better to follow the pattern there (a few other examples).

That is:

1. Rather than having a list for libraries you should use one {1:library_foo Library [foo]} section level heading for each library. That way you get a quick and efficient clickable library index on the LOCAL CONTENT of landing pages.
2. In each section you should be using the {!modules } directives so that the module synopses get spliced in.

Also I think you can drop page from the Documentation page heading and the section should simply contain the list of toplevel .mld page (it would be nice if we could specify a synopsis with an @ directive at some point).

(That way the automatically generated page gives a good blueprint on a structure to enrich to get to landing pages similar to those mentioned above)

[EmileTrotignon]

We agree your example should be using brr's index.mld page, instead of this, right ?

I am not sure what you mean by this, but my PR only concerns automatically generated package landing pages. Brr's index.mld is accessible by the first link in that page.

It would be nice not to link to this auto-generated page when a handmade index exists, which is the case in brr.

Later I think it could be nice to also have the package description from the opam file on that page.

Need for what this PR adds could also be reduced once we enable the sidebar in that page.

Rather than having a list for libraries you should use one {1:library_foo Library [foo]} section level heading for each library. That way you get a quick and efficient clickable library index on the LOCAL CONTENT of landing pages.

That's nice, thanks for the idea.

In each section you should be using the {!modules } directives so that the module synopses get spliced in.

That sounds good. My first attempt used {!Foo}, it does not work because of an issue in the driver and I now use {:Foo/index.html}. I think {!modules } will require resolving, I'll try to update the driver so that it resolves.

Also I think you can drop page from the Documentation page heading

Thats reasonable.

and the section should simply contain the list of toplevel .mld page

That is something we want, but it will be done in another PR.

(it would be nice if we could specify a synopsis with an @ directive at some point).

Thats another mission for Super-expression ! Jokes aside, its a nice idea that could go into the frontmatter, which is probably going to use tags (they will be special tags that trigger a warning if they are not at the start of the document)

[dbuenzli]

I am not sure what you mean by this, but my PR only concerns automatically generated package landing pages. Brr's index.mld is accessible by the first link in that page.

I'm confused. The landing page of a package should not be generated if there exists an index.mld page, it should use the latter which was carefully crafted by a human to guide users. (That's the way odig conventions work, use index.mld if there is one, generate one if there is none).

Otherwise it's one more click to get to my index.mld which will be more or less equivalent to your automatically generated page index which is not a very good user experience.

[panglesd]

I'm confused

Let me try to explain!

So, in the new design¹, we decided that pages and libraries will go in separate directory. One natural way to do it is to say that for a package p with some docs and libraries p and p.sublib, in the final hierarchy:

• the docs for package p will be in <html_root>/p/doc/,
• the API docs for library p will be in <html_root>/p/lib/p/,
• the API docs for library p.sublib will be in <html_root>/p/lib/p.sublib/

Moreover, the subhierarchy of mld files, below <html_root>/p/doc/, is the same as the one for html files, below <opam root>/doc/p/odoc-pages/.

One nice things about having libraries API and package docs in different directories, is that there is no constraint on the name of the directories in <html_root>/p/doc/: name clashes are impossible.

However, we did not want that removing a part of the url yields a 404, so we need to fill a few pages. In our example:

• <html root>/p/index.html,
• <html root>/p/lib/index.html,
• <html root>/p/lib/p/index.html,
• <html root>/p/lib/p.sublib/index.html.

(Note that in odoc 3, we do not plan to have any link to pages outside of <html root>/p/lib/<libname>/ nor <html root>/p/doc/: the breadcrumbs will stop before that.)

One way we could write those pages would be to make them redirect to the carefully crafted <html root>/p/doc/index.html.
In the reference driver, we decided to create actual pages to fill those gaps. (the <html root>/p/index.html page could be seen as an equivalent of the ocaml.org landing page for a package).

This is in my opinion not very important, since the reference driver is primarily a way for us to test new features we implement on real-life packages. It may be reused by ocaml.org, but not with the pages that are in this PR.

I hope this is clearer now!

Footnotes

1. Hopefully, this PR is ready enough to read if you want more technical details. ↩

[dbuenzli]

Thanks for the explanations.

Moreover, the subhierarchy of mld files, below <html_root>/p/doc/, is the same as the one for html files, below <opam root>/doc/p/odoc-pages/.

Not sure I got that.

In the reference driver, we decided to create actual pages to fill those gaps.

If you plan to reuse it for ocaml.org I don't think it is a good idea. It will lead to confusing user experiences (and confuse search engines about the "right" index page). Do not needlessly create content in the name of this dubious constraint:

However, we did not want that removing a part of the url yields a 404, so we need to fill a few pages

If there's no content, there's no content. At most redirect to the landing page, but 404 is fine too: users quickly learn up to where they need to delete an URL if the system is consistent.

Do not provide thousands of navigation pages and widgets it confuses the navigation for users and slows them down.

(Note that in odoc 3, we do not plan to have any link to pages outside of <html root>/p/lib/<libname>/ nor <html root>/p/doc/: the breadcrumbs will stop before that.)

Not sure what you meant by that. In my opinion the breadcrumbs should all stop at the package landing page (which is unclear to me where you plan to put it)

(the <html root>/p/index.html page could be seen as an equivalent of the ocaml.org landing page for a package).

Note that these pages are confusing as hell and provide an extremely bad user experience. Take a package like brr if you include the part on docs you are subjected three time to the same content. This an extreme waste of a user's time.

I think this page should follow the way odig works: the landing page of your package should be the index.mld. The meta bureaucracy can be added on that page (like odig does) or on a separate page but the landing page should be the docs. The focus of docs.ocaml.org should be to show the documentation of a package not it's meta bureaucracy or READMEs. The meta bureaucracy is something you only every look at the first time you use a package after that it's a thousand times the docs, optimize for that. The current system is a waste of my clicks and time.

[panglesd]

In my opinion the breadcrumbs should all stop at the package landing page (which is unclear to me where you plan to put it)

The "landing page" (by that I mean the author-written file found in <opam root>/doc/p/odoc-pages/index.mld) is currently going in <html root>/p/doc/index.html.
So, in the case of a module (which goes in <html root>/p/lib/<libname>/), the module page is not a descendant of the "landing page".

If you put the content of <opam root>/doc/p/odoc-pages/ in <html root>/p/ (instead of <html root>/p/doc/), and you have a file in <opam root>/doc/p/odoc-pages/lib/<libname>/Module/index.mld, those files will clash.
(Maybe this is still preferable than the current solution, I'm just explaining a reason to put the pages in a doc folder).

If you plan to reuse it for ocaml.org I don't think it is a good idea.

As I said, if ocaml.org reuses this driver, it is only for building the docs (with the new features such as source code rendering). ocaml.org will decide how they want to deal with pages outside of the odoc hierarchy.

[dbuenzli]

So, in the case of a module (which goes in <html root>/p/lib/<libname>/), the module page is not a descendant of the "landing page".

I'm not sure you say this from an URL perspective or a navigation perspective (the breadcrumbs) but if it's the latter this is not good. An up from a library's toplevel module page should always bring you to the index.mld landing page.

You can put it in <html root>/p/doc/index.html if you wish so, though I wouldn't do it because it makes your URLs less hackable. I would expect to trim my URL to <html root>/p/ to take me to the index.mld content.

Basically it's ok to put the docs and the manuals and libraries in separate roots but index.mld should be the root of all this, it's notably the page where you list all the libraries and the manuals…

[EmileTrotignon]

I have pushed a commit, now the package landing page looks like this :

[dbuenzli]

I don't think it's useful to nest the libraries under API, less hierarchy !

[EmileTrotignon]

I don't think it's useful to nest the libraries under API, less hierarchy !

I think in that case it is important because it explain what is the difference between the above "Documentation for brr" link and this one. Maybe it would be less of an issue if instead of having a "documentation" link we had a list of toplevel pages with their titles. Of course in the case of brr we do not want that page to exist but that is another matter.

[dbuenzli]

Let's talk about package landing pages. I think a good package landing page has the following structure (see the links given earlier in this discussion for many sizeable real world examples):

{0 package xxx}

Summary of the package

{1:manuals manuals}

Lists of manuals and their purposes

{1:library_foo Library foo} 

List of modules via {!modules: } directives. If there are 
many of them possibly under {2: subsections} to enable 
efficient access via `LOCAL CONTENT` to subsets of modules.

{1:library_bar Library bar}

…

{1:possibly additional sections}

{1:metainfo Metadata information}

Automatically added by the driver

This gives you an extremely efficient 'LOCAL CONTENT' table of content to quickly access essentials part of a package from the landing page.

Now in case this is generated I think we should also have exactly this structure. Except that:

1. "Summary of the package" is the opam description file.
2. The list of manuals in the Manuals section is automatically generated from all toplevel documentation pages found in the docdir.
3. The list of libraries and toplevel modules is automatically generated from what can be found in the libdir of the package.

That way there is little difference between a suggested human written package landing page and a generated one and the generated one provides a blueprint to start a human written one.

[EmileTrotignon]

I agree with you, and I had approximately the same goals (I had actually given a look at the links you sent earlier).
However, implementation of all of that is not 100% straightforward. For instance, for now, I can't make {!modules ...} work. I'm going to keep trying, but in the meantime there is going to be no description.
I havent tried fetching the list of pages yet.
I have looked a little bit at a way to fetch the opam description, and did not find a way I could implement in 10 minutes, so I should give it a better look.

[EmileTrotignon]

{!modules ...} now work !

I am now going to try to make certain pages be just redirections

[EmileTrotignon]

I have added a redirection for some pages when a handwritten page is present. The implementation is a bit janky for now.