docs: Add Python to the generated Markdown tool doc

[wenzeslaus]

This completely revamps the generated header for tool documentation pages, and it adds Python syntax for the tool and Python-oriented parameter description. The Python part is similar, but not the same to what is generated by Sphinx autodoc. I tried to make like it while accommodating the metadata we have (key desc strings, label and description for flags), while working with the Markdown syntax (we generate pure and simple Markdown which is only later translated to HTML in generic way, not knowing about the context or language).

Most of the headings are removed for both CLI and Python, key-value parameters go before flags for both, and the idea makes heavy use of synchronized tabs.

Top of the page

Current	New: CLI	New: Python
Note what is (not) visible right at the beginning...	...and what is visible here.	Python has the same pieces with changes to syntax as appropriate.
(no language indicated)	Command line	Python (grass.script)

Generated example Python calls

gs.run_command("r.neighbors", input="name", output="name")
gs.run_command("r.watershed", elevation="name", accumulation="name")
gs.parse_command("v.db.select", map="name", format="json")
gs.parse_command("r.univar", map="name", format="json")
gs.run_command("r.slope.aspect", elevation="name", slope="name")  # with a local fix for rules
gs.run_command("r.stats", input="name")
gs.run_command("r.what.color", input="name")  # with a local fix for input
gs.run_command("r.category", map="name", output_format="plain")
gs.run_command("r.null", map="name")
gs.run_command("g.copy")  # the rules don't allow dynamic parameters like g.copy has

[echoix]

For the example part, since I imagine it being difficult to be right in all cases, would there be a possibility to override the example manually with a valid example?

[echoix]

Overall it's really nice, especially for the CLI one (are the flags still there in the case of the middle picture?)!

[wenzeslaus]

Adding more info to the metadata is possible. It would be useful at other places too. We can keep a list of ideas. I was thinking about overriding the run_command vs parse_command choice even for the general syntax block.

For this PR, we need to come up with rules which will give good results with the existing metadata. There is already a lot of info available. For example, the additional rules are not touched at all.

[hellik]

Very nice!

It is called

Command Line (bash)

though in winGRASS we don't have a bash, only a Windows command line

[neteler]

Very nice!

though in winGRASS we don't have a bash, only a Windows command line

Perhaps remove the "(bash)" as there are also other shells like zsh, etc. Or call it "(shell)"?

[wenzeslaus]

Command line seems good here. It should be shell-neutral. What about later, in the examples? There we often use Bash specifically. Note that the tabs are synced like you can see e.g. in GitHub doc.

The story for Python may also be more complicated with the examples section.

[wenzeslaus]

...it's really nice, especially for the CLI one

I'm interested, what do you like with the CLI and why you give Python a lower ranking?

(are the flags still there in the case of the middle picture?)!

The flags are there at the end. For Python, the short flags are treated in a special way, differently from how allowed values with description would be treated (think e.g. format), although that's what they are from Python perspective.

Flags in CLI and Python

[wenzeslaus]

Any suggestions how to show the type information in Python? ...or any of the other info.

r.neighbors: input, output, size, method, ...

g.copy: raster, raster_3d

r.watershed: memory

[echoix]

I'm interested, what do you like with the CLI and why you give Python a lower ranking?

I don't know, the way it was spaced out, not too much on the line that showed the name of argument, so it was kinda smooth visually to find the structure of what is indented.

But it's kinda similar when I look today.

The Python full signature is kinda hard to follow as it is jammed, but probably required to be shown exhaustively at least once, as source of truth.

[echoix]

The flags are there at the end. For Python, the short flags are treated in a special way, differently from how allowed values with description would be treated (think e.g. format), although that's what they are from Python perspective.

The flags layout for Python is quite easy to follow (visually) without even reading!

[echoix]

Any suggestions how to show the type information in Python? ...or any of the other info.

Using Python type annotations maybe? How does mkdocsstrings through griffe present it with markdown? Or even sphinx? https://mkdocstrings.github.io/griffe/
https://github.com/mkdocstrings/mkdocstrings

r.neighbors: input, output, size, method, ...

g.copy: raster, raster_3d

I don't understand what from, to is as an annotation with code markup.

r.watershed: memory

Memory in MB, where does it come from? Is it a Python module (tool) that has this info in the parser metadata?

[wenzeslaus]

Using Python type annotations maybe?

That's what I was thinking, but I started with what I saw in pandoc doc.

How does mkdocsstrings through griffe present it with markdown? Or even sphinx? https://mkdocstrings.github.io/griffe/ https://github.com/mkdocstrings/mkdocstrings

mkdocstrings may be something which we end up using for the Python part (but that's a migration for another time).

On the other hand, just having the same thing as what we will have in the source code makes sense. Also something readable by non-experts like list[str] vs Iterable[Str]. I just didn't do much research on this.

I don't understand what from, to is as an annotation with code markup.

Memory in MB, where does it come from? Is it a Python module (tool) that has this info in the parser metadata?

Both from,to and memory in MB are key_desc which is what goes after the key_name= in command line doc. See the current g.copy doc as an example. It would be nice to have that additional info there, but it is also very unspecific. from,to can be "old_name,new_name" in Python, but also ("old_name", "new_name"). However, for display tool, it would likely be "10,50" or (10, 50). On the other hand, memory in MB would be 300. This is another place to extent the tool metadata, but for now, this is what we have.

[wenzeslaus]

I added the generated Python examples to the description. While I hit some limitations, it is pretty impressive what we can do with the current tool metadata. I have two open questions here:

1. How the detailed description in Python should look like? (esp types, but also the key_desc, both discussed above)
2. Should command line also have an example and how that example should look like? (it should be easier to create than for Python)

Then I need to organize the code better and it will be good to go.

Assuming this builds, if you download the artifact from Documentation workflow, you should get the full version including addons.

[wenzeslaus]

Tuples of ints and floats in Python are no longer just the numbers and commas, but they are now strings (which happens to be actually how the underlying implementation works, but I'm made that choice mostly to keep the code simple):

For the types, I used the same info to show the type as tuple rather than int or float. It gets complicated when multiple values are allowed. The parameter can be tuple, list of tuples, list, or a comma-separated string. I don't show all the options a this point, but I show most.

For simple multiple, we don't have a way of expressing "this can have more than one item, but usually it has one item" versus "this is expected to be more than one item" versus "this is required to be more than one item". Consequently, I consider "multiple" to expect one item by default (the first case), so r.univar gives expected result (str as first type) while r.series is suboptimal (list type is only a second type listed).

g.copy - classic tuples

r.univar - classic multiple

m.nviz.image - long tuple

[wenzeslaus]

I added the generated example for the command line and used only str, float, int, list, tuple (possibly nested) in the Python types (but no Iterable).

This is now ready ready for review.

[echoix]

Separating union of different types with a comma, but also separating the elements of a tuple with a comma, and also using the comma in between the types, the key descr from metadata, and between the optional indicator is probably confusing, at least for me, especially on a same line. If we are to show Python types, we might as well present it using a valid Python syntax. (The latest Python syntax for types is using the pipe as the union operator (this OR that type)).

Generally, my opinion for reference docs is to have true information, even if incomplete, rather than more info, but could be false. I don't mind if a section is marked experimental and warns if something might not work.

[wenzeslaus]

Separating union of different types with a comma, but also separating the elements of a tuple with a comma, and also using the comma in between the types, the key descr from metadata, and between the optional indicator is probably confusing, at least for me, especially on a same line.

Now using Python's pipe to separate types. Key desc is not moved under "Used as". Optional indicator and tuple items are still both separated by comma, but tuple items are inside parentheses and the optional indicator is now newly in emphasis/italics.

Generally, my opinion for reference docs is to have true information, even if incomplete, rather than more info, but could be false.

As far as I can tell, it is pretty clear overall, although we were not really showing most of that info in the past or using in the doc.

I don't mind if a section is marked experimental and warns if something might not work.

The example piece is the one closest to being experimental. While the heuristic is relatively straightforward, it needs to be tested more (evaluated in the wild). Not enough to warrant marking it experimental.

[wenzeslaus]

This r.slope.aspect example shows how the new "Used as" works for the most common case, input and output raster:

This m.nviz.image example shows a long type description and several examples of "Used as":

This r.watershed example shows (short) flags and long flags, now more aligned with how other parameters are represented:

[wenzeslaus]

Here is a link to the artifact if that makes it easier for anyone to review.

[petrasovaa]

#5358 (md to man) will need to be updated.