Poll on usage of Langium in adopter projects

[msujew]

With our 3.0 release coming up fairly soon, we're planning a few breaking API changes. One of them might be #1258, which splits the LSP related services from the general Langium services. The goal of this change is to be able to bundle Langium based applications without the LSP code and the LSP related dependencies, thereby reducing bundle size.

This would roughly decrease bundle size by 100kb. However, this is only for the use case of using Langium without any editing services. Language servers using Langium will not change in size with this PR.

We're still evaluating whether this change benefits the community as a whole or only introduces unnecessary complexity to the project. For that end, we want to know from the community how they use Langium.

[jindong-zhannng]

While I am investigating how to reduce the bundle size, I happen to see this post. ✌️

In my perspective, I am 100% in favor of reducing bundle size as much as possible.

I am building a dsl-based web forms solution. Of course there are some parts in my project that don’t care about the bundle size. For example the cli for parsing and validating form definition files, and the vscode extension for helping people writing files.

But in addition, it also has a part that is bundle size sensitive. For example the dsl evaluator that will run in frontend.

Currently my bundle is 1M+ while it only requires a parser following this way. This size is totally unacceptable for me.

So my question is:

• Is it possible that the parser only bundle size can be less than 100k in future versions?
• If not, is there any other suggestions for optimizing size? For example, never imports langium directly in a size sensitive environment? And I'd better to serialize and deserialize ASTs by myself to control size?

[msujew]

Hey @jindong-zhannng,

thanks for the valuable feedback!

Is it possible that the parser only bundle size can be less than 100k in future versions?

Aside from the LSP use case, we have a few other dependencies. These dependencies (mainly chevrotain, our parser library) are fully minified ~120kb, so even without the language server dependencies, you're likely looking at a ~350kb package at minimum. However, when serving the bundles gzipped, I've found most of them to be between 100 and 200k. So it should be possible to get them to the user with minimal data transfer. Actually, when looking at all of it gzipped, the difference in the language server vs parser only use case is only ~30kb, not ~100kb as initially claimed.

If not, is there any other suggestions for optimizing size? For example, never imports langium directly in a size sensitive environment? And I'd better to serialize and deserialize ASTs by myself to control size?

1M+ of size sounds like a lot, even for Langium. v2 bundles should be minified at roughly ~600kb. Have you set the mode to production in the language configuration when building your language? Do you also set the minify flag on the esbuild compiler?

[jindong-zhannng]

Aside from the LSP use case, we have a few other dependencies. These dependencies (mainly chevrotain, our parser library) are fully minified ~120kb, so even without the language server dependencies, you're likely looking at a ~350kb package at minimum.

I believe your team did your best on optimizations. But as a library author (comparing to end users), the bundle size is critical for me, at least, for marketing purpose. So even ~350k is definitely good enough for real-world applications IMO, it's still unacceptable for a library. (You know, I have no chance at all to argue with end users that they can enable gzipping)

1M+ of size sounds like a lot, even for Langium. v2 bundles should be minified at roughly ~600kb. Have you set the mode to production in the language configuration when building your language? Do you also set the minify flag on the esbuild compiler?

This was measured without minify. The only reason I need to bundle up codes by myself is I wanna distribute a cjs export for my library but langium is pure esm now. So even I do bundle codes, I wanna give control of whether doing minify to my users.

---

Let's back to the size. I'm not a language engineering expert, I would like to ask you if it is possible to build the parser from scratch using other tool instead of langium (or chevrotain) to obtain a smaller size, but integrate it back to langium in some way to obtain the benefits of auto-generated LSP services?

[msujew]

But as a library author (comparing to end users), the bundle size is critical for me, at least, for marketing purpose.

Yeah, I get that. While I wouldn't necessarily say these things are out of scope for Langium (see for example zenstack which is using Langium as part of a library fairly successfully), Langium is a pretty powerful framework and designed to be consumed directly rather than transitively.

I would like to ask you if it is possible to build the parser from scratch using other tool instead of langium (or chevrotain) to obtain a smaller size

In theory, but currently not in practice. We have thought about it, for example to make it easier to create an ANTLR4 based parser. But there is nothing in the pipeline right now to enable that. Note that chevrotain was mainly chosen for two features: Its error recovery and speed. Replicating both of these features in a hand-written parser is quite non-trivial.