Rich Text

[cooper-joe]

Rich text

Summary

Rich text functionality is needed. Several apps allow users to enter text in different contexts. Some examples:

• interpretations/comments in all analytics apps
• notes in Capture app
• dashboard items in Dashboards
• custom forms in Capture app

These and many more user inputs need to support rich text. Different contexts will require different rich features, so the user interface will need to be flexible.

Discussion points

There are several main discussion points:

• how to store/manipulate rich text? (html, markdown, etc.)
• how to expose rich text controls to the user? (Toolbar, inline special characters)
  • should we use a third-party library? (draft, quill, etc.)
• what rich text features will we include? (formatting, links, images, mentions, etc.)

What are we doing now?

Rich text is already available in Analytics interpretations. Should we use this as a starting point?

The user interface

The user interface could consist of two parts:

• toolbar style, always available buttons that create and adjust formatting when clicked
• inline markdown-style formatting/markup

The type of user interface to use will depend on the context. Dense, information rich UIs intended for tech-comfortable users will benefit from inline markup. A user with low tech literacy will benefit from Word-style formatting controls and WYSIWYG editing.

It is possible to combine these two interfaces and allow them to be used interchangeably via a toggle (as used in JIRA). Whether we do this, and how we do it, is a topic to discuss.

Next steps

To move this discussion forward, @dhis2/front-end please contribute experience, opinions and ideas for the above points. Thanks! :)

I have started work on some initial ideas for the user interface. This discussion will inform those designs, especially 'What rich text features will we include?'.

[cooper-joe]

Rich text functionality

Features required, based on my experience and previous discussion and feature requests, sorted into must-haves and nice-to-haves.

Formatting

Must-haves

• bold
• italic
• underline
• text color (pick from palette of x colors)
• bullet list (multi-level)
• URL

Nice to haves/Maybes

• font selection
• text size (small, regular, large, XL)
• text background color (pick from palette of x colors)
• number list (multi-level)
• image
• quote
• emoji
• code block/inline code
• predefined styles:
  • header level 1
  • header level 2
  • header level 3
  • body

DHIS2 specific

Must-haves

• user mentions (@a_dhis2_user)

Maybes

• analytical object link (e.g. search for a chart via @ and link it without providing the URL)

[HendrikThePendric]

Rich text is already available in Analytics interpretations. Should we use this as a starting point?

I think this would be a great idea, because the current rich text component in d2-ui:

• Is a markdown based editor, like Slack/ Github (this means it uses inline special characters)
• From what I could tell by a quick inspection, the formatting logic is completely decoupled from the UI, so it would work with ui-core and d2-ui components already

To me that approach makes a lot of sense.

[HendrikThePendric]

Also, I think it is important that in the end it is important that we store the formatted text in some sort of markdown format, not HTML (I remember seeing discussions about that in the past but can't provide a reference). And if we do so, I think that the following features would be a bit unconventional:

• text color (pick from palette of x colors) (MUST HAVE)
• font selection
• text size (small, regular, large, XL)
• text background color (pick from palette of x colors)

I guess anything is possible, and I am not an expert in Markdown syntax, but I think it would be convenient if we could do without the features above. @cooper-joe is text-color perhaps something we could do without as well?

Also, I think it is important that in the end it is important that we store the formatted text in some sort of markdown format, not HTML (I remember seeing discussions about that in the past but can't provide a reference). And if we do so, I think that the following features would be a bit unconventional:

* text color (pick from palette of x colors) (MUST HAVE)

* font selection

* text size (small, regular, large, XL)

* text background color (pick from palette of x colors)

I guess anything is possible, and I am not an expert in Markdown syntax, but I think it would be convenient if we could do without the features above. @cooper-joe is text-color perhaps something we could do without as well?

Yeah I've also never seen those features in markdown before. Markdown-based editors is what I've seen pretty much exclusively lately for rich text editing. I think I remember some xml-based ones in the past, but that was quite a while ago. It makes sense to me to stick to what the common variants of markdown allow.

[cooper-joe]

I guess anything is possible, and I am not an expert in Markdown syntax, but I think it would be convenient if we could do without the features above. @cooper-joe is text-color perhaps something we could do without as well?

Text-color and text-size are often requested features, especially for communicating with text on dashboards. This doesn't mean we have to include them, but I think we need good reasoning not to. The reasons you stated above are already convincing.

Markdown would be simpler to implement, store and use. It is limited, of course. I am concerned that choosing markdown would cut off any non-markdown functionality in the future. Many of our users will be used to text-editing via Word, so no 'text size' or 'text color' is a major omission. Using HTML to get access to these few features seems like overkill and opens up rich text editing to 'you-can-do-anything' editing.

What about extending markdown with a few of our own features? Has anyone got any experience with that? (For example, a custom tag like regular text ::size-L:: Large text ::size-L::)

[HendrikThePendric]

Apart from the benefits you mention, using a markdown format would mostly be beneficial from a security perspective.

I don't have any experience with creating custom markdown features, but it will be possible to implement technically. There are many markdown "flavours" and then we'd be creating our own. And by doing so, plus having a WYSIWYG UI, we would make things more familiar for users that are used to working in MS Office etc. However, at the same time, we might make things more complicated for people who are accustomed to working with markdown directly....

Also, it'd be more complicated for us. Which might lead to bugs, slower updates, etc. I'm not sure if there are existing solutions that allow customisation. I'll see if I can find anything.

So there's something like this: https://www.npmjs.com/package/markdown-it. It allows for extending the syntax, so if we want extra syntax this would an option. In my experience markdown parsers are fairly heavy though, that's a downside (lazy loading it would be nice). And the more we stick to the defaults, the easier it'll be to maintain, the more stable, etc. Does it make sense to start out simple for example, and add more if it seems absolutely necessary along the way?

[cooper-joe]

There are many markdown "flavours" and then we'd be creating our own. And by doing so, plus having a WYSIWYG UI, we would make things more familiar for users that are used to working in MS Office etc. However, at the same time, we might make things more complicated for people who are accustomed to working with markdown directly....

In my anecdotal experience, our users are far more comfortable with MS Office than markdown. Also, provided we started at the base markdown-flavor and provided a toggleable 'raw' non-wysiwyg editor, markdown preferring users shouldn't suffer.

Does it make sense to start out simple for example, and add more if it seems absolutely necessary along the way?

Yes, I agree that ordinarily that is the best approach. However, in this case we need understand what the hard requirements are I think. If we start simple (markdown) and try to add text-color at a later date, but it turns out to be too difficult, we would either need to drop text-color or start again with HTML. If text-color is a hard requirement the only option is starting over. So, it looks like to make a decision here we need to find out whether these non-markdown features are absolutely required. I'll ask some questions and see what I can find out.

[HendrikThePendric]

If text-color is a hard requirement the only option is starting over.

I don't think this is a likely outcome, since the lib that @ismay found is extendible/customisable.

On the other hand, if you would be able to assess whether font-size/color is a requirement upfront, that would be very helpful.

[varl]

Rich text is already available in Analytics interpretations. Should we use this as a starting point?

I think this would be a great idea, because the current rich text component in d2-ui:

* Is a markdown based editor, like Slack/ Github (this means it uses inline special characters)

* From what I could tell by a quick inspection, the formatting logic is completely decoupled from the UI, so it would work with ui-core and d2-ui components already

To me that approach makes a lot of sense.

I've ported the d2-ui-rich-text component to ui-widgets a while back: https://github.com/dhis2/ui-widgets/tree/port-rich-text.

The branch probably needs updating though.

Our implementation uses Markdown-It by the way: https://github.com/dhis2/d2-ui/blob/master/packages/rich-text/src/parser/MdParser.js#L1

We already deviate from Markdown in d2-ui-rich-text (slightly), so technically we have a DHIS2-flavored Markdown that we can expand on.

[varl]

Priorities

I disagree that text color should be on the "must have" list. Here's a revised proposal based on the implementation we have now as v1:

Must-haves (v2)

* bold
* italic
* underline
* bullet list (multi-level)
* number list (multi-level)
* headings (level 1-6)

Nice to haves (v3)

* image
* URL
* quote
* preformatted text

Maybe in the future

* font selection
* text size (small, regular, large, XL)
* text color (pick from palette of x colors)
* text background color (pick from palette of x colors)

I think that any font and color customization is out of scope for the foreseeable future, there are a lot of complexities lurking in here related to accessibility and other features that rely on stylesheets (e.g. Dark Mode, High Contrast Mode) to operate, so that is why I would put them in the "maybe later" pile.

A final note is that "emoji" support was considered a "must have" when the current rich text implementation was built, so that could go under "DHIS2 Must haves".

wrt/ Storing in HTML

During the June 2018 release we debated HTML vs. Markdown, and settled on moving away from storing HTML in the database and towards "plain-text storage".

One reason we did so is because we did not want to rely on dangerouslySetInnerHTML as it opened us up to XSS attacks[1].

We should use a sanitizer like https://github.com/cure53/DOMPurify in our rich text component in combination with dangerouslySetInnerHTML. However, I'd much rather sidestep that additional complexity everywhere though. I doubt that even if we re-open this discussion with backend tech leads they will want to go back to storing HTML in the database.

[varl]

So, it looks like to make a decision here we need to find out whether these non-markdown features are absolutely required. I'll ask some questions and see what I can find out.

We can support a hybrid model of HTML-in-Markdown which is commonly used around the Web, so even if we go for Markdown-ish formatting now we aren't locked out of implementing support for text color/size/background, etc.

(Given that we are allowed to store HTML in the database again, which in either case of HTML and Hybrid we would have to do.)

[cooper-joe]

Thanks for the clarfications @varl. Looks like the logical starting point is to build upon the custom Markdown-It markup we use today, following the priorities you outlined above. I'll work from this base point in the design specs.

The complexities of supporting color attributes suggests that this is indeed a maybe in the future feature. The component design will allow for these features, but I won't spec them in the first instance to avoid confusion.

Also, +1 to emoji support being a must-have. The current support in d2-ui-rich-text is a good starting point and I think any expansion can follow that pattern. I think the major difficulty here will be agreeing which emojis to support.

[cooper-joe]

I have another question about rich text functionality:

Looking at other libraries/frameworks, it seems the most common technique is to use contentEditable on an element and allow the editing to take place in there. As I understand it, that means we would not be able to use the TextArea ui-core component when using rich text features? Is there an obvious, proven concept/structure we should follow here?

This relates to the design stage I am currently working through. I'm facing with questions like:

• should a rich text toolbar be standalone, or can it be embedded inside a TextArea?
• would RichTextArea be a new component?
• could a rich text toolbar apply styles inside other components (e.g. a normal InputField?)