SRT 32 character line limit doesn't take into account wide/fullwidth characters

[ryan-lp]

Characters in Chinese, Japanese and Korean are twice the width of characters in the Latin alphabet. As such, applying the same 32 character line limit in SRT files to Chinese as you would do to English would effectively allow Chinese SRT lines to be twice as long as English lines and may overflow the viewing window. Probably for Chinese, the character limit should be half of 32, i.e. 16.

If we allow for multilingual transcripts (#367 ), then there may be a mix of Chinese words and English words in the same sentence, and so it would make more sense to specify that each Chinese/Japanese/Korean character counts as 2 characters when computing the 32 character line limit.

For more information, see: https://en.wikipedia.org/wiki/Halfwidth_and_fullwidth_forms

[daveajones]

Is this a reference to something in the namespace transcript spec or something in the SRT spec itself?

[ryan-lp]

It is in reference to the podcast namespace spec:

https://github.com/Podcastindex-org/podcast-namespace/blob/main/transcripts/transcripts.md#srt

As far as I'm aware, there is no official specification for SRT on maximum line length, although various organisations have their own guidelines. The general idea is the same, to put just enough text on the screen that it appears for at least 1-2 seconds, and doesn't exceed two lines. The namespace's standard of 32 characters is possibly a little bit narrower than most other recommendations I have found for English, but too wide for Chinese/Japanese/Korean due to the double-width characters.

[ryan-lp]

Looking across the web for existing guidelines, I found the following page from JBI which also arrived at the same 16 character line limit:

https://jbilocalization.com/tips-subtitling-captioning-japanese/

It would be relatively straightforward to amend the spec to halve the character limit for languages with double-width characters, although I think it would be more flexible when considering multilingual transcripts to instead count each double-width character as 2 characters. That way, if a single line contains a mix of both Chinese and English words, it should still add up to the correct line length.

[daveajones]

Linking this on Podcastindex.social for more visibility, along with #407. https://podcastindex.social/@dave/110083279500496922

[jamescridland]

How's this?

"SRT files are for use in simple closed-captions or can be parsed for display on a website. SRT lines should be limited to 32 visible characters, to ensure they are visible on a wide variety of screen sizes without alteration. SRT files are more widely supported than other formats, and are highly recommended."

"JSON files are for use in more advanced circumstances, and contain timing for each word's start and end."

[ryan-lp]

How's this?

"SRT files are for use in simple closed-captions or can be parsed for display on a website. SRT lines should be limited to 32 visible characters, to ensure they are visible on a wide variety of screen sizes without alteration. SRT files are more widely supported than other formats, and are highly recommended."

This doesn't address the issue at hand (maybe you posted it on the wrong issue?). I would suggest wording along these lines:

"SRT lines should be limited to 32 characters with each full-width character (Chinese/Japanese/Korean) counting as 2 characters."

[ryan-lp]

As another reference, here is Netflix's limits.

They give a special character limit for Chinese, Japanese and Korean, and then they have an "everything else" category. The CJK character limit is roughly half that of the "everything else" category, however Netflix doesn't actually specify what should happen when using a mix of scripts in a single line, so the wording I proposed above (or something like it) would better handle that case (not only for multilingual podcasts, but also for the Japanese language itself whose writing system already mixes 4 different scripts including Latin.)

[ryan-lp]

SRT lines should be limited to 32 visible characters
This doesn't address the issue at hand

Ah, unless the word "visible" is intended to indirectly do all the work of accounting for full-width characters, not by mentioning them explicitly but just by implying that anything "invisible" would be bad. Although I find the wording a bit problematic.

On Netflix and Teletext, we can tell subtitle publishers to make sure their text will be visible precisely because the Netflix and Teletext font proportions and layout are specified, and visibility can be determined from that. But in podcast apps, different developers are free to use different screen layouts and font proportions. SRT publishers can't simply be told to make sure their text will be "visible" in any app, and then somehow be able to derive that the character limit for CJK languages should be 16. I think it's got to be the other way around, since SRT publishers only deal with character limits, not font proportions and layouts which vary from app to app. So we need to give SRT publishers a concrete character limit, and THEN it would be up to the app developers to ensure visibility.

[jamescridland]

Yes, I meant "visible" as exactly that - what is visible on the screen once you've decoded it.

"SRT files are for use in simple closed-captions or can be parsed for display on a website. SRT lines should be limited to 32 visible characters on-screen after decoding, to ensure they are visible on a wide variety of screen sizes without alteration. When using some alphabets or characters, that may mean an SRT file is longer than 32 characters before decoding. SRT files are more widely supported than other formats, and are highly recommended."

Does that work better?

So - 💩 is (I think) three characters - U+1F4A9 - but displays as one character on-screen after decoding. Or maybe it's two characters. But it displays as one character on-screen.

That gives us a hard and fast rule - character limits, not worrying about the proportions of text and trying to calculate the difference between the length of 'm' and of 'i'. That's up to the app to deal with (and the app knows that were they to use fixed-width text then everything would be a maximum of 32 characters and the same physical length once decoded.

[jamescridland]

I'd probably add: it's likely that Netflix shows bigger text for complex characters like Chinese and Korean, to aid in comprehension. My memory from watching SD television on a CRT screen in Hong Kong would suggest that's the case: the resolution wasn't capable of showing text as small as the characters we use.

[ryan-lp]

I'd probably add: it's likely that Netflix shows bigger text for complex characters like Chinese and Korean, to aid in comprehension. My memory from watching SD television on a CRT screen in Hong Kong would suggest that's the case: the resolution wasn't capable of showing text as small as the characters we use.

But that's not a side note or an afterthought, that is actually THE whole purpose of this issue :-)

Compare these two lines, the first in English, the second in Japanese:

Hello and welcome to this week's
こんにちは、そして今週のあなたの好きな番組のエピソードにようこそ

Both of these lines contain exactly 32 characters each. The 32 character limit suffered from the same sort of oversight that often happens with CJK languages (Chinese, Japanese, Korean) (see this), although these 3 languages are a well-recognised exception in typography, which is why subtitle guidelines tend to have a general rule that applies to most languages, and then tend to have language-specific exceptions for these specific languages.

It's not as though we have 100s of languages and therefore we have 100s of potentially different character limits to address each language. No, actually latin script has been highly influential throughout the world's languages, with many languages simply borrowing and extending the latin alphabet, and even for scripts that don't follow the latin alphabet, their typography has followed latin conventions in that they have adopted a similar average character width to latin. The exceptions are very few, and so we can list these exceptions explicitly to make it absolutely clear. That is to say, we don't need to come up with some general and vague wording that could apply to any language in the future, we can actually list out the exceptions right now and make it clear and unambiguous.

This issue is not about how 'i' and 'm' may have different widths in proportional fonts, that's ultimately irrelevant when designing a publishing guideline for subtitles because the guidelines are based on the averages that are appropriate for the particular language. And in English, lines of text with similar character counts will tend to have roughly similar widths due to the way averages play out, even though proportional fonts allow for theoretically worst case extremes. This is somewhat addressed in most guidelines which advise having some margin space on either side when rendering, which means that there is some extra room for proportional fonts to grow.

This issue is also not about unicode encoding and decoding. The number of bytes used to encode a unicode codepoint does not determine the visible width of that character, and those bytes are not "characters" by modern character counting conventions (on UNIX, see wc -m vs the vestigial wc -c). I think if you start bringing in this low level terminology into what should be a really simple spec, it will only confuse the matter. When programmers implement the spec, what they are going to do when counting how many characters are in a line is they're going to use the standard unicode string length function in their respective modern programming language that supports unicode. And for the above two example lines, they're going to get a count of 32 characters of English, and 32 characters of Japanese. What's then needed is a clear spec that simply says that actually these particular languages have a different character limit which is XYZ, and aside from those special cases, every other language has a character limit of 32.

(In my original proposal, I was also trying to address the case of multilingual podcasts. But even then, I think we should avoid going to the level of byte encodings which has nothing to do with rendering, and would confuse the matter. In a UNIX terminal of 80 columns by 24 rows, originally designed for latin script in ASCII encoding, that originally meant an 80 character line limit. But when this was extended to support CJK languages, that meant a line limit of 40 CJK characters, or 80 latin characters, or when a line contains some combination of both kinds of characters, each CJK character occupies the same amount of space as 2 latin characters. If we are to have a special case for certain languages like this, rather than define a special character limit of 16, we could instead specify a special width factor of 2, where that number could contain decimal places.)

[jamescridland]

Please: I gave a (hamfisted) example of what the spec should say. Here's where you can do similar.

Propose what the spec should say, and we can all agree to it. This isn't a space for a long argument, this is a space for "here's what the spec should say in my experience - everyone agree?"