Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exported files should only have git diffs when assets change #16

Open
pnadolny13 opened this issue Apr 22, 2022 · 0 comments
Open

Exported files should only have git diffs when assets change #16

pnadolny13 opened this issue Apr 22, 2022 · 0 comments
Labels
enhancement New feature or request

Comments

@pnadolny13
Copy link

pnadolny13 commented Apr 22, 2022

The CLI is working as expected but the assets that are produced sometimes have diffs without making any changes. Its challenging for the developer to have a clean development + deployment workflow using git. I'm not totally sure what pieces of this are CLI related or Superset related so I can make other issues in the Superset repo if thats needed. I'm also happy to give more feedback, contribute, or test any changes out!

The ideal workflow that I'm hoping to implement:

  1. clone git repo containing Superset assets
  2. spin up local Superset instance
  3. import Superset assets to local instance
  4. make changes in the UI
  5. export Superset assets to local directory
  6. optionally make edits to the exported assets if custom behavior is needed like using Jinja templating
  7. commit changes to git branch
  8. get code reviews from teammates. Potentially have staging instances to share with reviewers.
  9. merge changes which kicks off a CICD pipeline that pushes the updated assets to the production instance of open source Superset or Preset

Challenges:

  1. Chart assets have an ID in their file name and that ID changes sometimes.
  2. Related to number 1, when importing charts their IDs look like they might be changing. Maybe they get imported in a different order. It ends up causing all the chart assets to have diffs when nothing has changed in them. Also the dashboards that reference those charts get diffs because they have chart ids that need updating to match the new ids.
  3. Dashboard assets have a list of charts by position that doesnt seem to be consistently ordered. Git shows diffs in the charts but really its just a reordering.
  4. When renaming a chart it becomes a totally new asset. So the diff looks like a file was deleted and another was added. This isnt ideal when you update the name and also make a change to the contents because the code reviewer wont be able to view the changes because git thinks the whole file is new. It might be challenging since the file name is the chart name but ideally a chart name update would just be an update to a line in the yaml vs file deletion and addition.
  5. [Nice to have] If I cloned my repo and imported my assets into a local instance, then delete a few charts in my dashboard using the UI then I export again the assets still persist in my directory. It would be up to the developer to make sure they remove the stale assets. Having a way to detect a deletion would be ideal so the developer would have less of a chance of leaving stale assets in the repo.
  6. [Nice to have] When importing my assets to the production instance, I would want to delete any assets that no longer exists in the git repo. Having a way to detect deletions on import would be great.

cc @betodealmeida

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants