Untyped expressions in generics/templates

[metagn]

Have been writing this issue for hours so there might be mistakes

Most of generics (and templates?) are typed by default but presumably we still need the untyped prepass either via an explicit untyped escape hatch or for stuff like when statements which cannot be resolved. I don't think this would be much different from the old compiler's untyped prepass at least in principle, any problems with it that are not solved by generic typechecking would probably have the same solutions in either compiler.

Some problems can occur when mixing typed and untyped expressions however.

when sizeof(T) == 4:
  let abc = 123
else:
  proc abc(): string = "xyz"

let def = abc # ???

We cannot declare a single symbol abc here because it would be declared in both a (let) and a (proc) definition. However we can detect that something named abc was declared in the current scope since both branches declare it (just one is enough really), and consider abc as a typed expression, just with the type untyped. This would also be the type of direct untyped expressions, like when sizeof(T) == 4: 123 else: "abc". Then we can still declare def as a unique let symbol, just also with the type untyped. Will get to the type untyped later.

This is mostly syntax level however. If the compiler has no way of knowing something was declared, it can't give it a type either.

when sizeof(T) == 4:
  foo() # macro that declares abc
else:
  bar() # ditto

let def = abc # undeclared identifier: abc
untyped:
  let def = abc # works
let ghi = def # also works

It is also sensitive to scope:

when sizeof(T) == 4:
  block:
    let abc = 123
else:
  block:
    let abc = 456
let def = abc # undeclared identifier: abc

---

Templates/macros that declare symbols could also take advantage of this, via some {.injects: abc.}. This could be inferred for templates, and could make use of identifier construction for either, i.e. template foo(x: untyped) {.injects: `x`.}, or whatever form identifier construction ends up taking.

template foo(x: untyped) =
  let `x Value` = 123 # infers `foo` to be {.injects: `x Value`.}

untyped:
  foo(abc)
let def = abcValue

If foo is overloaded then we could include the injected symbols of every overload, no need to be too restrictive. There is also the possibility that an outside unconsidered overload would match.

There is also the case where the arguments declare the symbols, in which case we can maybe use something like injectsOf.

template foo(body: untyped): untyped {.injectsOf: body.} =
  body # again this would infer injectsOf anyway

foo:
  let abc = 123
let def = abc # works but abc still has the type `untyped`, in contrast to if `body` was a `typed` parameter

Finally, there is the case where the arguments to the template itself use injected symbols (if we implement untyped parameters). Then we can use usesInject and usesInjectsOf pragmas specific to the parameters but these don't have to be different to injects and injectsOf, maybe they can both be named just inject and injectFrom. This could also be inferred.

template foo(defs, body: untyped): untyped =
  let abc {.inject.} = 123
  defs
  block:
    let def {.inject.} = 456
    body

# inferred signature:
template foo(defs {.usesInject: abc.}, body {.usesInject: [abc, def], usesInjectsOf: defs.}: untyped): untyped {.injects: abc, injectsOf: defs.}

Maybe we wouldn't need to manually analyze the AST at the end of every untyped context for these and could make it part of the untyped type i.e. untyped {.injects: abc.} but this might be unnecessary complexity.

---

The type untyped would propagate in expressions similar to unresolvedness as in #252, however it entirely replaces the type of the expression and acts as a bottom type. It's already implemented that a call with any argument expression with type untyped causes the entire call to become untyped. Probably dot expressions and subscripts and most other "compound expressions" would work similarly, however subscripts still need to capture [] symbols as in original Nim and builtin subscripts need to be considered first for them at typing time. Maybe we can introduce a new node kind for this.

Unlike unresolved generic types which are substituted, untyped types have to be deleted entirely during instantiation in a way that matches the original parsed AST, i.e. it can only be deleted in a way that acts like auto or never added in the first place. This means untyped generally should not be a child of another type, anywhere we construct a type based on the type of child expressions should check for and propagate untyped. But they are fine if matching existing types, i.e. Foo(field: abc), [123, abc, 456], let x: int = abc.

Something important is that type expressions can also be untyped.

untyped:
  type T = int
type U = T # untyped
var x: foo(T) # untyped

While the type of the symbols are untyped, we cannot replace the original type AST in the symbol definition with (untyped). We could maybe keep a set of which symbols are untyped when typechecking generic procs and query that when getting the type of a symbol. Hopefully the places we need to do this are limited, since untyped symbols should stop typechecking. Worst case, maybe we could reuse unresolved types in #252 but with base type untyped and without any generic parameter information.

We could also make use of the untyped type for erroring nodes/syms/types that need the above behavior or we might need to handle error nodes as types.

[Araq]

Sounds mostly reasonable but quite hard to implement & maintain. Here is my proposal:

• Develop the type checker as we need it for new type-checked generics, not too many special rules for untyped etc.
• Introduce a .untyped pragma for templates and generic procs that replicates Nim v2's behavior to a degree that we can consider "so good it's bug compatible".

Else we end up with a complex set of rules that are different from Nim v2's set of rules and nobody can ever update his codebase...

If this means that I cannot use crazy when complies(foobar) stuff within a type-checked generic, so be it. As long as the escape hatch is easy enough to get.

[Araq]

Since then I changed my mind and UntypedT already propagates upwards, not unlike C++'s template parameter dependent expressions work.