Support for Motoko language

[ccelik97]

I'd really appreciate if you add support for Motoko language of Internet Computer too https://sdk.dfinity.org/docs/language-guide/motoko.html
It's getting popular over time as they've made to the public source release of the Internet Computer platform.

Also I'm not sure if it's a good thing to make an addition like this (if not I can create another issue for this) but what about an optional, generic support for all languages that use regular {} blocks etc?

[leodevbro]

I just need a good Motoko tokenizer which has input of text and output of an array of opening/closing positions of blocks.

position is like -> line 70, column/char 19, or global index like: 743.

We can just search for curly brackets in the text, but it must be more advanced, because commented bracket must not be identified as opening or closing, it must be ignored.

I have some problems with time currently, but I hope I will try it soon.

[ccelik97]

Found here about it and I wonder if language server's availability helps with your extension.
There's also the Motoko Grammar in the docs if that helps too but if not feel free tell me how else I can help. It's been a while since I last time played around with language syntaxes with ANTLR etc in university but I hope there's something I/we all can do to help :D

[ccelik97]

On Dfinity's forum I found that it's only BNF (Backus-Naur Form) with the addition of optional (...)? phrases. I hope this helps with supporting it.

From this comment https://forum.dfinity.org/t/motoko-grammar-explanations/5658

[leodevbro]

Unfortunately I don't have the knowledge of how to use language grammar/syntax (or language server) to make a tokenizer for that language. In Blockman I used as many prepared tokenizers as I could find in the internet, but it seems difficult to find some other tokenizers, one of them is Motoko tokenizer, also Ruby tokenizer. Maybe someone else can help us to find or build these tokenizers.

Also, since the Blockman is written in TypeScript, any tokenizer should be written also in TypeScript (or JavaScript), otherwise maybe it will be problematic to implement it in Blockman. So, if we find a Motoko tokenizer written in let's say C++, then maybe we cannot integrate it in Blockman because the entire Blockman is built with TypeScript and JavaScript.

[ccelik97]

If you don't know how to do these then what about having a basic mode for the usual () {} [] "" '' `` <> etc? The languages Blockman doesn't have specific tokenizers for them (or any other piece of a text document really) would be covered to a good degree by doing so.

[leodevbro]

@ccelik97, Hi, I think Blockman can work with Motoko language file, if you install the Motoko extension of VS Code.

what about having a basic mode for the usual () {} [] "" '' `` <> etc?

The main problem is that, sometimes brackets are inside comments, or inside strings, and if we just apply a trivial algorithm to grab all the brackets, then the entire fire structure blocks can become super incorrect and super messy. And it needs pretty complex algorithms (or regex) to detect which bracket is block bracket and which bracket is neutral bracket.

[ccelik97]

Looks OK. Once I have a Motoko installed again, I'll give the extension a try again.