Fix/syntax error calling multiline call return value

[JR-Mitchell]

What?

This PR introduces the ParseLang enum and parse_lang field to the teal parser.
This allows different behaviour when parsing depending on whether a file is a .lua file, a .tl file, or a .lax.tl file (more on this below).
This is then used in this PR to ensure that lua code which calls functions on a new line, or after a multiline string literal, parse correctly.

Why?

Primarily, as a proposal to resolve #865.

Language syntax differences
#748 added logic to the parser which creates a difference in parsing behaviour between Lua and Teal.

In Lua, the following is valid syntax:

print
("hello world")

and results in "hello world" being printed when run.

However, in Teal, this new casting syntax is permitted:

(myVal as MyType):method()

Which introduces situations where it is not possible to determine whether a statement like this is intended as a new statement, or a call to a function or method on a previous line, such as:

local record MyType
    getComparator: function(self: MyType): function(other: MyType): MyType
end

local a = {}
local b = {}
local c = (a as MyType):getComparator()
(b as MyType):getComparator()

The last 2 lines here could either be interpreted as

local c = ((a as MyType):getComparator()(b as MyType)):getComparator()

or as

local c = (a as MyType):getComparator();
(b as MyType):getComparator()

#748 resolves this ambiguity by making it so that Teal does not allow function calls on new lines.
However, this meant that the teal parser was no longer compatible with the parsing of Lua, meaning that if any Teal code required any Lua code that used this syntax, it would fail.
For example, this broke the base64 Lua library in teal.

This PR aims to handle that the parser is used to parse 2 different languages which now have subtly different parsing rules by passing a flag in the ParseState for which language's parsing rules are being used, so that Teal- or Lua-specific behaviour can be conditionally applied when parsing.

Migration of Lua code to Teal

This PR also introduces a third ParseLang, lax-tl, representing files with a filename ending .lax.tl.
This is my proposed approach to resolving some of the comments mentioned in #743 about gradual migration of Teal code to Lua code - in particular:
> Of course, .lua files with partial annotations are not proper Lua files

and
> Requiring .lua files into .tl files practically compels you to rewrite the code unless you write a .d.tl file for every .lua file and maintain them alongside your .lua code

This PR allows for .lua files to be renamed to .lax.tl files, which will now be parsed with the Teal syntax as opposed to Lua syntax, but will have lax type checking performed on them (as if they were .lua files).
This suggested approach would hopefully make it easier to migrate Lua code to Teal, as you simply rename your .lua files to .lax.tl files and begin to add type annotations, and then once they are fully migrated, drop the .lax from the filename, as opposed to the previous approach, which would require an additional .d.tl file and also leave you with a .lua file that isn't actually valid Lua.

Other notes

I made the parse_lang parameter optional, primarily for backwards compatibility (so that any use of functions such as tl.parse_program would not break due to the extra added argument), and also because there is one situation where I wasn't sure what to pass (line 7012 in tl.tl).
However, I am aware that making this non-optional may lead to more robust code, as the type checker would ensure that a non-nil argument is always passed for it.

Please let me know in general whether you think this PR is a good or bad approach to resolving this issue, and specifically whether you think I should rewrite so that parse_lang is non-optional.

[github-actions]

Teal Playground URL: https://874--teal-playground-preview.netlify.app

[pbdot]

Instead of adding a file extension, which then tooling also needs to be made aware of, perhaps it would be cleaner to use something like a compiler annotation in the Lua source, which the tl picks up on. Annotations could be used for other purposes as well, including existing compiler options. For example, https://typescripttolua.github.io/docs/advanced/compiler-annotations. There are also pragmas, https://github.com/teal-language/tl/blob/master/docs/pragmas.md#pragmas

In short, our practical goal for pragmas is to allow for handling compatibility issues when dealing with the language evolution. That is, in a Teal codebase with no legacy concerns, there should be no pragmas.

[JR-Mitchell]

Yeah, that's a fair point. I'll rework this PR to remove the .lax.tl stuff I added in the next couple of days.

[hishamhm]

I don't think this needs it because we only load well-known and well-behaved code here (the compatibility chunks listed below in add_compat_entries), but since these are Lua chunks, it could be a good idea to explicitly pass "lua" as the parse_lang, if only for documentation sake.

[hishamhm]

@JR-Mitchell it's late here so I won't trust my own review at 1:30am and will take a closer look at this in the coming days, but overall it's looking good!

Regarding your discussion with @pbdot above, I wonder if the way to go would be to add a #pragma lax on like we do for arity. I've avoided so far it for the sake of keeping the .lua and .tl boundaries more separate and encouraging people to just go all the way with typing, but if the current state of things makes it more difficult to migrate, then perhaps the pragma would be the best bridge?

[hishamhm]

looking good! I think we can just remove that TODO (by either keeping parse_lang implicit or explicit there, your call!)

[JR-Mitchell]

@hishamhm thanks!
I've updated so that parse_lang is explicitly "lua" in that case, I think that makes the code overall clearer.
I also made the parse_lang non-optional in default_env_opts, since it was explicitly passed at all the call sites anyway, and probably should be specified whenever used.

[JR-Mitchell]

Btw I don't have permissions to merge to this repo, so someone else will have to do that for me if they're happy to

[hishamhm]

Will do when I'm back at it -- I'm taking a holiday break but will get back to it early Jan

[hishamhm]

@JR-Mitchell I squashed some of the commits and merged the PR manually! Thank you!!