Add missing items on bundle export and import

[vinayvenu]

Need:

Different tickets and cards raised for every item mentioned below.

Issue:

The following items are currently missing in the bundle export.

• translations
• documentation - the form elements in the App designer don't show the documentation, though the exported particular form.json has that mapping to documentation - seems to be working
• dashboard description and custom dashboard filters - all info Included in export file
• icons - subject_type and report_card : icons3Key and icon file are included in downloaded bundle
• old rules and rule dependency

The scope of this story is to add these into the bundle export and import

Analysis:

• Though export happening in most cases, import not handled or has issue.
• Subject type icons import not working within the same environment as well
• Though report_card icons are not used currently in mobile app, since we are doing for subject_type should be easy to do. Handling of subject_type icons can be done similar to extensions.
• Skip fixing for old rules and rule dependency if it is more effort - more than an hour effort.

Acceptance criteria

• The missing items are present in the exported bundle
• Missing items are successfully created / updated when importing bundles
• Documentation Bundle export / import works

Out of scope:

• External system config should not be exported since this will have secret information

[mahalakshme]

Duplicate card with more details for "documentation": #490
For transalations: avniproject/avni-webapp#972

[himeshr]

Test cases to cover:

1. Download bundle from org (source) which has all of these setup
   • Organisation specific Translations for atleast 2 languages
   • OldRules and Rule-Dependency
   • SubjectTypes with Icons
   • CustomDashboard with Description
   • CustomDashboard - Sections with Description
   • CustomDashboard - ReportCards with Icons
   • Documentation for any Form's FormElementGroup / FormElement
     Confirm that all config is captured as part of the bundle content.
2. Create a new org (target) and upload the bundle, all config should be setup exactly as specified in the source org
3. Make changes in the source org config and then download bundle again, ensure that all changes are reflected in bundle file
4. Upload the new bundle to target org and verify that all changes are reflected
5. Download bundle for target org and do a diff with source org bundle to confirm that content only varies only in uuid or positioning only

[petmongrels]

• If we defined the BundleFolder then we should use the enum defined there instead of using subjectTypeIcons, reportCardIcons, etc as direct string.
• VideoService.saveVideo method, not sure what scenario it is handling and the reason for change (as the code is more complex now)

[himeshr]

@petmongrels Had to make the change to init video entity using the contract uuid as seen in commit ce9bd8f, because, during new org init using first time bundle upload, if the source org had a video entry, then that would not be found on target org, on search using uuid.
It would then result in failure later, when we execute videoService.populateVideo() call before saving video entity.

[AchalaBelokar]

@himeshr then I Login with AchalaB@tmp try to upload the bundle it is showing one error downloaded that error sharing with this

SNCU.zip

errors-SNCU.csv

• Tested with sncu which contains all translation.

• Tested with different different subject icon.

• [x]

[himeshr]

AchalaB@tmp

This is a bug caused due to the way we are handling Default Groups "Everybody" and "Administrators".
Have pushed a commit to handle it correctly. Please test after applying the server fix to the test environment.

[AchalaBelokar]

• @himeshr After uploading the same bundle on same org another error is coming

errors-SNCU (2).csv

[himeshr]

@AchalaBelokar Try the groupDashboards again. Please ensure that you use a fresh bundle downloaded from prerelease for this.

[mahalakshme]

@AchalaBelokar what are the test cases done?

• Did u check custom dashboard filters?
• How did you test old rules?

[AchalaBelokar]

• For older rule I tried with Calcutta kids export and import and check weather the rule is working fine in the new organisation.
• this is the Calcutta kids zip file which I uploaded

Calcutta Kids (3).zip

I got this error file.

errors-Calcutta Kids (3).csv

[mahalakshme]

@AchalaBelokar test other cases as well simultanousely, what about custom dashboard filters?

[AchalaBelokar]

• I tried with custom dashboard filter which is present power getting the same error
  PoWER (3).zip
  errors-PoWER (3).csv

[himeshr]

• I tried with custom dashboard filter which is present power getting the same error
  PoWER (3).zip
  errors-PoWER (3).csv

The issue encountered for Power and CK is due to code changes in avniproject/avni-server#1328.
Neverthless, fixing it here.

[himeshr]

@AchalaBelokar please hold off from testing..
the code change isn't working as expected, some compiler overOptimization issue.

[himeshr]

@AchalaBelokar I've resolved the issue and triggered deploy of the build to pre-release environment.
To test the fix, generate a new bundle from ck or power org and use that to setup a whole new org say "powerBundletest".

This will not work with old bundle, as the current design is such that, we process bundle upload in the same file sequence as we create the bundle. Since old bundles were created with files in wrong sequence, we will still get errors for them.

I've created a separate card to work on making the bundle upload agnostic of bundle creation file order here
#709.

[AchalaBelokar]

• created a org Ach Power bundle test which is new org

• calcutta kids download that bundle and upload that bundle other org which is working for old rule..

• changes are done source org dwonload the bundle and upload on destination org

[AchalaBelokar]

• deleted some data from AchalaB@acs download the bundle and upload it to AchalaB@acd
  this is the

Ach calcuttakids source.zip the zip file occurring this error

errors-Ach calcuttakids source.csv

[himeshr]

• AchalaB@acs

Hi Achala, on debug, i found that the following difference in individualRelationship name is the root cause of the bundle upload failing for org with user AchalaB@acd.

There is a difference in the relationship "name" between orgs acs and acd. And we currently use only the name field during individualRelation upload.

Made code changes to primarily use uuid for this purpose.