Multiple Widgets - Speed Issues

[gavinr]

Report from @thejones #61 (comment):

At work we used the generator and Grunt and everything fell apart when we got past a couple of widgets. It was simply too slow and only gets slower. Put a few widgets into a folder and let grunt run and you will soon see this. So, The Grunt workflow as-is only works, IMO, if you are developing single widget/single repo.

I would personally like to see the workflow change to better support multiple widgets or make it easier to develop one widget per branch without maintenance headaches. We "solved" this at work by using npm scripts and creating symlinks so that the folders no longer copy/sync. Our app just had a link into the dist folder. This works so well for us I would highly recommend looking into something similar. We got it to work on Windows 7 so anything is possible...

At some point it will be unreasonable to expect the Grunt task to transpile and copy files for all the widgets in your project (100 widgets for example, I think we can all agree would be unreasonable). But I agree that the generator should support more than "A couple" widgets. I think we need to:

1. Figure out the current performance (how much time does it take to transpile and copy 3 widgets? 4 widgets? etc)
2. let's come up with a benchmark that we want to meet (example: The Grunt task should compile and copy the files within ~3-6 seconds for up to 3-5 widgets.) , then
3. implement code changes (most likely in the Gruntfile) to get closer to that benchmark.

[thejones]

One part that I left out is that we did experiment with accepting a "current widget" argument when launching the Grunt task. I would say that even if 20 widgets are in source control most people will be editing 1 at a time. So, having an option that says only compile and sync widget_x would also work. It has been a long time since I tried this and I had some issues with the file glob patterns not being respected. It is an option that might be worth pursuing.

[davewilton]

My project got really slow. I found part of the issue was the fact that using copy to put all the files into the dist folder meant they were all updated (because the date changed), and then they all got copied over to the WAB directory. I got much better results using sync for both copying the resources (html etc) from the source directory.

My build task is set up to create a zip for deploying to portal rather than packaging the whole wab site. Simple

You can see how before it updates all the files, this gets more and more slow the bigger the project is

After it is only the widget that has been touched

I don't have time to pull this at the moment, but you may want to consider trying the below for a large TS project

module.exports = function (grunt) {
    'use strict';
    grunt.loadNpmTasks('grunt-sync');
    grunt.loadNpmTasks('grunt-contrib-watch');
    grunt.loadNpmTasks('grunt-contrib-copy');
    grunt.loadNpmTasks('grunt-contrib-clean');
    grunt.loadNpmTasks('grunt-ts');
    grunt.loadNpmTasks('grunt-bump');
    grunt.loadNpmTasks('grunt-contrib-compress');

    // eslint-disable-next-line @typescript-eslint/no-var-requires
    var appDir = 'C:/WAB/wab_2_19/server/apps/9';
    var stemappDir = 'C:/WAB/wab_2_19/client/stemapp';


    grunt.initConfig({
        sync: {
            wab: {
                verbose: true,
                files: [
                    {
                        cwd: 'dist/',
                        src: ['**'],
                        dest: stemappDir
                    },
                    {
                        cwd: 'dist/',
                        src: '**',
                        dest: appDir
                    }
                ]
            },
            dev: {
                verbose: true,
                files: [
                    {
                        cwd: 'widgets/',
                        src: [
                            '**',
                            '!**/*.ts',
                            '!**/*.d.ts',
                            '!**/*.js.map'
                        ],
                        dest: "dist/widgets"
                    }
                ]
            },
            build: {
                verbose: true,
                files: [
                    {
                        cwd: 'widgets/',
                        src: ['**',
                            '!**/*.ts',
                            '!**/*.d.ts',
                            '!**/*.js.map'
                        ],
                        dest: "build/widgets"
                    }
                ]
            }
        },
        ts: {
            dev: {
                // specifying tsconfig as a boolean will use the 'tsconfig.json' in same folder as Gruntfile.js
                // for the dev version we want source maps
                tsconfig: true,
                options: {
                    additionalFlags: '--outDir dist',
                    inlineSources: true,
                    inlineSourceMap: true
                }
            },
            build: {
                // specifying tsconfig as a boolean will use the 'tsconfig.json' in same folder as Gruntfile.js
                // no source maps for build
                tsconfig: true,
                options: {
                    additionalFlags: '--outDir build',
                    inlineSources: true,
                    inlineSourceMap: true
                }
            }
        },
        watch: {
            main: {
                files: [
                    'widgets/**',
                    'themes/**'
                ],
                tasks: [
                    'ts:dev',
                    'sync:dev',
                    'sync:wab'
                ],
                options: {
                    spawn: false,
                    atBegin: true,
                    livereload: true,
                    interrupt: true
                }
            }
        },
        bump: {
            options: {
                files: ['package.json', '**/widgets/**/manifest.json'],
                push: false
            }
        },
        compress: {
            build: {
                options: {
                    archive: function () {
                        var pkg = grunt.file.readJSON("package.json")
                        return 'build/' + pkg.name + "_" + pkg.version + '.zip'
                    },
                    mode: 'zip',
                },
                files: [
                    {
                        src: '**/*',
                        cwd: 'build/widgets/',
                        expand: true
                    }
                ]
            }
        },

        clean: {
            'dist': { 'src': 'dist/*' },
            'build': { 'src': 'build/*' }
        }
    });

    // copy all the ecl over when the task starts. If you make updates to 
    // ecl then stop and start this task
    grunt.registerTask('default', ['clean:dist', 'watch']);

    grunt.registerTask('build', ['clean:build', 'bump', 'ts:build', 'sync:build', 'compress']);
};