docx to html numbering

[andrew-danieli-fj]

Hi,

I'm currently using Pandoc (v3.7.0.2 on windows) to convert an application manual in docx format to html so that it can be used as an online help system. This was originally done using "Word" save as filtered html, however this produces cluttered html, which can be a pain to match the previous conversion (given changes in the version of Office used over the years). So far the results are promising, but I'm stuck with a section numbering issue.

The word document uses H1 - H6 for numbered section headings and numbered text paragraphs and H1 - H4 are fine with the default format (e.g. 1, 1.1, 1.1.1, 1.1.1.1) . H5 and H6 are actually formatted in word as ordered lists, H5 uses a. b. c. etc, whilst H6 uses (1) (2) (3) etc. Given how numerous H5 and H6 are used through the document, I don't really want to reformat it there, and was wondering if the Pandoc default numbering scheme could be overridden with a Lua filter to handle the formatting here?

The other issue I've found is that when a parent section header is skipped, the numbering for the child starts at zero.

Heading Level	What I was expecting	What Pandoc Produces
H1	1	1
H2	1.1	1.1
H4	1.1.1.1	1.1.0.1

If H3 was inserted between H2 and H4, the H4 numbering would be what I was expecting (and what is seen in the original word document). Is it possible to use a filter to reset the H4 numbering to start at 1 when the H3 isn't present?

Any suggestions would be appreciated. My only issue is that I'm new to Lua development, so will be learning as I go when developing any filters. Thanks in advance.

[rnwst]

Could you attach a simple example DOCX exhibiting the issues you describe? At the moment I find it hard to understand the issues you are facing, but I think an example doc might fix that.

[andrew-danieli-fj]

Regarding your question about your filter functions being executed more than once, a filter function for an element is run once for each occurrence of that element in the AST. So if the document has 3 Headers, then the Header filter function will be run 3 times.

Thanks. I get the element level (e.g. "header") but for blocks, I'm not sure how that works internally. When running my filter above, I expected "BlocksPass1" and "BlocksPass2" to be called once each, but instead, they are both called multiple times, with only the last one seeming to do any processing. Is that because the block type doesn't match what I'm looking for?

[rnwst]

The Blocks filter function is executed everytime there is a list of Blocks in the AST. The top-level document will have Blocks, so the function is executed at least once, and then those blocks typically will contain more lists of Blocks (the OrderedList element for example contains a list of a list of Block elements, so an OrderedList with 3 items will have the Blocks filter function executed another 3 times). You can learn more about Block AST elements here: https://hackage.haskell.org/package/pandoc-types-1.23.1/docs/Text-Pandoc-Definition.html#t:Block

[andrew-danieli-fj]

I've managed to find some time to look at this again, and I'm struggling to work out how I should move sections of the AST around? In this instance I need to make one ordered list the child of an element of another ordered list.

In the example filter code above, I'm directly referencing both the parent and child lists within the blocks variable of the filter. As the error being thrown by my filter doesn't indicate a line number, I assume that I've managed to corrupt the AST doing this, and Pandoc or the logging library can't process the contents of the blocks variable.

Any suggestions would be appreciated. As a note, I'm using VSCode to develop the filter with the logging library to help with debugging. So far I haven't found a good example of setting up VSCode to debug a Pandoc lua filter, such that I can step through the code and set breakpoints.

[andrew-danieli-fj]

Well I've got a bit closer by using the clone() function to snapshot the two parts of the AST into local variables (which I cast as OrderedList).

   local parent_list = blocks[i-1]:clone()
   ---@cast parent_list OrderedList
   local child_list = blocks[i]:clone()
   ---@cast child_list OrderedList

   -- to get the last element of the parent list (which is itself a list of blocks containing just one entry) I've used
   local last_child = parent_list.content:at(#parent_list.content)
   ---@cast last_child Blocks

The issue now is if I try to extend last_child with child_list or concatenate child_list onto the end of last_child the call fails with a similar error.

   last_child:__concat(child_list)
   -- throws bad argument #1 to '__concat' (table expected, got Block)

   last_child:extend(child_list)
   -- throws bad argument #1 to 'extend' (table expected, got Block)

I'm not sure what I need to do to append the child_list (OrderedList) to last_child.

As a refresher this is what I'm trying to achieve (shown in markdown)

# The following is a nested ordered list

1. First Item
2. Second Item
3. Third Item
   1. Indented Item
   2. Indented Item
4. Fourth Item

where last_child = 3. Third Item and child_list = 1. Indented Item...

which produces the following pandoc structure that I'm after.

(#) pandoc Pandoc {
  blocks: Blocks[2] {
    [1] Header {
      attr: Attr {
        attributes: AttributeList {}
        classes: List {}
        identifier: "the-following-is-a-nested-ordered-list"
      }
      content: Inlines[13] {
        [1] Str "The"
        [2] Space
        [3] Str "following"
        [4] Space
        [5] Str "is"
        [6] Space
        [7] Str "a"
        [8] Space
        [9] Str "nested"
        [10] Space
        [11] Str "ordered"
        [12] Space
        [13] Str "list"
      }
      level: 1
    }
    [2] OrderedList {
      content: List[4] {
        [1] Blocks[1] {
          [1] Plain {
            content: Inlines[3] {
              [1] Str "First"
              [2] Space
              [3] Str "Item"
            }
          }
        }
        [2] Blocks[1] {
          [1] Plain {
            content: Inlines[3] {
              [1] Str "Second"
              [2] Space
              [3] Str "Item"
            }
          }
        }
        [3] Blocks[2] {
          [1] Plain {
            content: Inlines[3] {
              [1] Str "Third"
              [2] Space
              [3] Str "Item"
            }
          }
          [2] OrderedList {
            content: List[2] {
              [1] Blocks[1] {
                [1] Plain {
                  content: Inlines[3] {
                    [1] Str "Indented"
                    [2] Space
                    [3] Str "Item"
                  }
                }
              }
              [2] Blocks[1] {
                [1] Plain {
                  content: Inlines[3] {
                    [1] Str "Indented"
                    [2] Space
                    [3] Str "Item"
                  }
                }
              }
            }
            listAttributes: ListAttributes {
              delimiter: "Period"
              start: 1
              style: "Decimal"
            }
          }
        }
        [4] Blocks[1] {
          [1] Plain {
            content: Inlines[3] {
              [1] Str "Fourth"
              [2] Space
              [3] Str "Item"
            }
          }
        }
      }
      listAttributes: ListAttributes {
        delimiter: "Period"
        start: 1
        style: "Decimal"
      }
    }
  }
  meta: Meta {}
}

[andrew-danieli-fj]

Right I've managed to solve the creation of nested ordered lists.

When I needed to add a child list to the last element of its parent, I just needed to use

-- to get the last child of the parent
local last_child = parent_list.content:at(#parent_list.content)
---@cast last_child Blocks

local last_child_len = #last_child
last_child[last_child_len+1] = child_list

-- then update the master blocks list
blocks[i-1] = parent_list
-- then remove the original copy of the child
blocks:remove(i)

and when combining sibling lists at the same level I used the following

parent_list.content:extend(child_list.content)
-- then update the master blocks list
blocks[i-1] = parent_list
-- then remove the original copy of the sibling
blocks:remove(i)

From the above I created a couple of local functions nestChildren(childkey, parentkey) and combineSiblings(siblingkey), which I needed to call inside my main function in the following sequence

nestChildren("UpperAlpha-Period","LowerRoman-Period") -- Loop through blocks making H8 lists children of H7 lists
combineSiblings("LowerRoman-Period") -- Loop through blocks combining H7 sibling lists

nestChildren("LowerRoman-Period","Decimal-TwoParens") -- Loop through blocks making H7 lists children of H6 lists
combineSiblings("Decimal-TwoParens") -- Loop through blocks combining H6 sibling lists

nestChildren("Decimal-TwoParens", "LowerAlpha-Period") -- Loop through blocks making H6 lists children of H5 lists
combineSiblings("LowerAlpha-Period") -- Loop through blocks combining H5 sibling lists

Now I just need to fight with CSS to get the levels of indentation for each list type to match the original word generated html.