Middleware support to allow arbitrary ts.CompilerHost changes

[toriningen]

This PR adds an ability to intercept arguments of ts.createProgram in an extensible way — with mechanism resembling Express middlewares.

By hooking methods of ts.CompilerHost before they are passed further, it's possible to do things I find nice:

• first class importing of non-TypeScript modules — e.g. for compiling GraphQL typedefs on the fly, or for using CSS files without bundling and import replacement.
• virtual modules — because file system access is completely incapsulated by ts.CompilerHost, it's possible to add arbitrary resources that TypeScript compiler will see as files.
• module resolution — enabling Yarn PnP without waiting for TypeScript team to agree on integration.
• etc.

I didn't include any code for implementing these scenarios, as I believe it makes more sense to keep them separated from ttypescript, as I assume ts.CompilerHost interface might change. However, it's quite trivial to implement an ad-hoc ts.CompilerHost delegate with desired interface using this mechanism.

I've yet exposed only createProgram middleware, with an intent for future expansion, as this will allow to provide composable interface to arbitrary TypeScript APIs.

[toriningen]

Also, for convenience, until this PR is reviewed, merged and published, I'm hosting https://github.com/toriningen/ttypescript-only to enable installing of this forked package through NPM.

[cevek]

Why createProgram can be optional here?

[toriningen]

In future, if more methods to intercept are added, it might be undesirable to require a module to implement no-op forwarders for all methods that are there. Also requiring every method would break existing middleware modules, as their export signature would no longer match with what ttypescript expects.

I'd suppose that in future this interface would become something like:

export interface Middleware {
    createProgram?: CreateProgramMiddleware;
    resolveModuleName?: ResolveModuleNameMiddleware;
    transpileModule?: TranspileModuleMiddleware;
    // etc...
}

And implementors would be able to choose what functions are they interested in.

[cevek]

I thought about this earlier, but in more wide context something like about more low-level plugin, which can combine many small transformers within one plugin. And program transformation runs through all plugins before file transformation stage.

And maybe in future also add macro transformation which can transform files before compiler starts check files

export default {
    transform: {
        program: (program, userconfig) => program,
        file: [(program, userconfig) => sourceFile => sourceFile],
        bundle: [(program, userconfig) => bundle => bundle],
        afterBuiltin: [(program, userconfig) => sourceFile => sourceFile]
        afterDTS: [(program, userconfig) => source => source],
    }
}

[toriningen]

What you are describing seems to be a more high-level take on the same idea. I'd say it relates to this PR as program transformer factory relates to raw factory. E.g. it's possible to implement transform with just raw, just transform is more convenient for those who don't need such level of control (and responsibility).

Similarly, what you describe can be an extension to middleware type. Welp, it could be even said that the whole transformation that is being done now in ts.createProgram hook imposed by ttypescript can be reworked as a middleware, but I thought it would be a way large change for a single PR.

[blaenk]

And maybe in future also add macro transformation which can transform files before compiler starts check files

I'm in need of something like this. I need to preprocess some files (simple regex replace) before they actually get parsed (otherwise they won't parse). From my cursory investigation it seems like I would do this by providing a custom CompilerHost with my own getSourceFile() implementation which would read the file, preprocess it, then pass it on to the compiler.

It seems like this PR would enable that, except that I still want to perform a before transformation, would this PR as-is require me to define them in separate plugins? Would there be some way for one (middleware) to pass data to the the other (before transformer)?

[cspotcode]

I need a way to plug into diagnostics so that I can filter diagnostics for certain files. I can accomplish this via a language service plugin, but those are not run in tsc and tsc --watch.

Essentially, I also need a way to override getProgram, but not to add transformers, to filter the results of getSemanticDiagnostics and other diagnostics methods.

[nonara]

@cspotcode ts-patch supports altering diagnostics. Not sure about your use case, but I believe it should work

[cspotcode]

Thanks, I'll check it out. I'm trying to understand why ttypescript and ts-patch use different approaches to patching the compiler.
It looks like ts-patch writes modifications to the filesystem, whereas ttypescript patches in-memory. Are there special considerations when using either tool with yarn2 PnP?

Are there any standards for writing a diagnostics filter that works for both CLI invocations and language service? For example, is there a language service plugin that will load the same set of diagnostics filtering plugins, so I can configure once and get the same filtering behavior everywhere?

[nonara]

The main two reasons for ts-patch doing a filesystem patch are 1) reducing drag during compile time and 2) making configuration for tooling easier. Tooling is set by default to load from the typescript directory, so direct modification eliminates the various different configuration instructions needed for tts.

I haven't actually used or looked too deeply into yarn2 yet. It is possible to use ts-patch with yarn1, however—although they make it a little trickier because of cache rewriting. An example is in the crosstype repo.

Are there any standards for writing a diagnostics filter that works for both CLI invocations and language service

This would be great, but unfortunately not. This, the yarn issue, and the patching mechanism is something I've been thinking about for awhile now. I also saw the proposal to rename ttypescript.

Ultimately, I'd really like to rewrite ts-patch with some key differences. I decided to feel out interest here: #113