UserForm files are not being created through "Launch Editor"

[sashayar13]

Check the attached screen recording

Screen.Recording.2024-05-29.at.11.34.02.AM.mov

[sashayar13]

[sashayar13]

Appears that files are actually created - but I had to leave the BPMN editor to find it out

[madhurrya]

It takes a long time for the form editor to open. https://drive.google.com/file/d/1WuBR85m3PGAFJg1PjEPv3ddTrCD3DRtV/view?usp=sharing

[madhurrya]

@danfunk noticed that some times when opening an existing form also takes a long time to load the content.

[danfunk]

As @madhurrya pointed out, it is creating the files, it is just taking a devastating 40 to 60 seconds to complete it with no indication in the UI as to what is happening. These are VERY small files it is creating - just three empty json files. Seems better to figure out why things are slow than to add some sort of minute long spinner.

The same long delay happens when you create a new json file (20 seconds), markdown file (16 seconds), or DMN file from the file menu on the process model page under the files tab. It isn't as evident in the ui, but it's always around 20 seconds to complete.

It is not slow when you modify a file that already exists.

Here is one call made to the backend:
https://dev.app.spiff.status.im/api/v1.0/process-models/misc:qa:forms:guest-form1/files/test4-schema.json
it took 20 seconds to complete.

The call is being made to a function _create_or_update_process_model_file in the process_models controller. It's doing the following things:

1. spec_file_service.update_file
2. spec_file_service.get_data
3. it generates a sha/hash of the file data
4. it commits and pushes to git
5. If it is a new file (and it is) it calls DataSetupService.save_all_process_models()

That DataSetupService walks the whole process model file tree and rebuilds our cache for all files and all process models everywhere. This would take a long time. I wonder if it is possible to run this on a sub-set of the tree since we know where we are writing the files? Is this an option @jbirddog , @jasquat ?

[jbirddog]

That last step was added to fix the use case of downloading/deleting/uploading a bpmn file that may be a call activity (so its has the proper references in the editor). Don't see why we couldn't skip that step if its not a bpmn file.

[madhurrya]

@danfunk
Now that loading time issue is fixed.
Still there is a slight delay there. If we can improve that also it'll be nice from the user experience side. But not sure whether it's possible.
Please check this video https://drive.google.com/file/d/10-n6ug93V5EzIDq6x4VKK6VIvPbpZaTZ/view?usp=sharing
And this one was from timetracker (before these code updates) which was much faster. https://drive.google.com/file/d/1zXropDnTX8MWmWhjYBgayIjsDUI_EZkc/view?usp=sharing
Don't have a much idea about the configs of each server and whether it's dependent on them.

[burnettk]

It will get significantly faster when we combine the three form related json files into one. If the original issue is fixed, I vote that we close this and wait for that other issue.

[madhurrya]

@sashayar13 Please see whether it fixes your issue. It looks OK to me.

[sashayar13]

@madhurrya, I agree that it takes a considerable amount of time to move to the next step - display the Form Editor. @burnettk, if we can't improve the performance right now - shall we add a progress icon showing that smth is happening? It's the same as we display for interstitial pages, etc. when a user has to wait

[burnettk]

yes, a progress indicator would be good here. i think that might be a 20 minute task. will take a look. one thing in the back of my mind here is that we are intending to combine the three form-related files into one, which would likely have a pretty significant impact on the amount of time it takes to initialize the form files. we had actually started on that work in the context of sartography/bpmn-js-spiffworkflow#77 (comment), but then set it aside temporarily to look at task data sizes, performance, and messages. as that work concludes, we could get back to combining the files and making this faster.