Contracted Braille Input Issues

[slannon97]

NVDA doesn't translate text as you type in contracted braille until you press the space bar or dots 7-8 simultaneously. This results in behavior similar to braille input in iOS 10 and earlier. If you are typing too quickly, the translator will miss characters. If you are trying to edit words, inserting a character causes the character to be translated. You must use the letter sign before the letter to get around the translation issue. For instance, if I wanted to correct table to tables, I would have to move the cursor to the end of table and type dots 5-6 followed by an s. If I simply tried to type an s at the end of the word, it would become tableso. In short, I want the seemless braille typing and editing experience that is present in JAWS, current versions of VoiceOver, and other note taker devices.

I am using NVDA 2017.4 on Windows 10 version 16299.248 with a Classic Focus 40 Blue. This is the model with the whiz wheels.
I realize contracted input is relatively new, but I figured I would post this. If this is a known issue, never mind.

[LeonarddeR]

@slannon97: Thanks for your report

@jcsteh: as you are the author of contracted braille input, could you please give your opinion about this report? I assume some of these limitations are known to you, so it would be good to have sort of an idea about what could be fixed and what are possible difficulties or shortcomings (e.g. in liblouis).

[Adriani90]

cc: @jcsteh maybe some input would help here to start future work on this.

[jcsteh]

This results in behavior similar to braille input in iOS 10 and earlier.

One major difference is that NVDA doesn't have a timeout before translating like iOS <= 10 did. It only translates when you end a word, not after, say, 2 seconds.

If you are typing too quickly, the translator will miss characters.

First of all, let's separate out this issue. If you're experiencing missing characters, this is not related to the way NVDA buffers words for braille input. To be clear, I'm not saying you're wrong; I'm saying this should be filed as a separate issue and a debug log provided.

Technical discussion for potential implementers:

The fundamental problem here (and why I went with waiting for a full word to be typed) is that entering a braille cell can modify characters before this cell, and if you're editing, even characters after the entered cell. For example (in English):

• If you type "t" (⠞), this will translate to "that", but if you then type "e" (⠑), the new translation is "te". The "hat" has to be replaced with "e".
• If you're editing the word "altogether" (braille "alt" ⠁⠇⠞) and you insert a letter sign (⠰) before the "a", the new translation is "alt". Even though the cursor was before the "a", you have to remove the "ogether", which is not even at the cursor; it's after. So you can't just simulate backspace presses to deal with that.

This means you need some way to quickly, reliably replace the entire word at the cursor every time the user enters a cell. This is complicated because NVDA has to support a lot of different APIs. While some APIs have clear ways to do this, others probably don't. Looking briefly, I don't think UI Automation provides any way to do this. I suppose you could select the word at the cursor and type over it, but then you'd need to suppress those selection events somehow to prevent spurious output.

Furthermore, different apps have wildly differing definitions of what a "word" is, and because not all APIs are offset based, we can't have our own standard definition of a word for braille input.

To make things more complicated, you need to display (on the braille display) the raw braille cells for the word at the cursor, not the translated text of the word being modified. For example, if you type "thi" (⠹⠊) and you then type "s" (⠎), the word will be changed to "this", but when that's forward translated, you'll get ⠹, so the ⠊⠎ goes missing. If the user intended to type something afterwards (e.g. "thisaway"), that is a big problem. And this again comes back to the different definitions of a word and the lack of offsets. How do we know which part of the displayed text to replace with raw braille cells? I guess one way of dealing with that problem is to render the word at the cursor as a separate TextInfo chunk, similar to how we render the selection as a separate chunk now.

Hopefully, that covers some of what needs to be considered here.