switch from marked to unified, remark and rehype and the performance considerations

[vlrevolution]

I notice you switched to remark in v4. I am personally also using that same stack that I put together for my own app in a very similar manner to how carta now functions. My app gets the markdown streamed in from LLM which is an important use case going forward. This however causes performance issues because marked is a blocking rendererer, it uses a single thread and the best we could do is offload it from main thread to another thread in webworker but the performance in webworker is even worse than the main thread... The marked takes 100% of cpu of the main thread when parsing, which is not ideal for webapps. Using webworker is a potential solution but introduces slower performance for some reason, when dealing with huge amount of markdown (such as code that in turn has to be rendered by shiki), it is not a very nice or scalable experience. I was thinking to pick your brain a bit about marked, as it promises to be non blocking & performant parser. What are your thoughts on this?

It sucks because unified, remark and rehype provide so much good stuff but when you eventually run into the performance issue trying to render large amount of streamed complex markdown the cpu goes to 100%💥

Maybe you have already thought about this, given you went with marked in the first place

[BearToCode]

Hello! I was a bit confused by the first paragraph because you talk about the new version with remark causing issues and then you mention marked, so I guess you are actually referring to remark, right?

I don't have experience with rendering streamed Markdown. Naturally re-rendering everything for every token that gets added will cause performance issues... I think that most chatbots just manually append new content to the current tag, since they ofter just use basic markdown syntax. Maybe you could try rendering content only when a line/paragraph has finished being streamed, so that it only runs a couple of times. You could slowly fade the new rendered content each time so that it appears smoother. I think Bard does something like this.

[vlrevolution]

Actually using a webworker to offload the work is a good solution! I just tried it yesterday and managed to not freeze the UI at all! Also yeah debouncing/throttling of rendering is important as some LLMs spit out chunks at 1ms

[vlrevolution]

I mentioned marked because it is non block in its design as opposed to remark, but remark with rehype and unified is definitely a better dev xp