Docker guidance and/or tagging should default users to latest stable release

[sfirke]

There are constant Github issues and Slack posts from new Superset admins who are trying to configure it for the first time and are struggling because the default Docker config is to run the latest-dev image. This is true even in the docker-compose-non-dev.yml file.

Using the latest nightly build has several problems, e.g., unique and temporary bugs that are hard to troubleshoot and a waste of time, trouble downgrading back to 2.0.0 once a user realizes they want to run a stable release. For example, here's a case where someone has problems because they deployed with latest following "official docs."

If the default Docker workflow instead used the latest stable release, that would be big step to reducing barriers to using Superset. Two ways I see to do that are:

• Change the docker-compose-non-dev.yml file to point to the image tagged 2.0.0. Quick fix but could fall out of date if not added to the release process.
• Change how image tags are used. Have latest point to the latest official release and latest-dev point to the latest dev version of that release. As in the other option, the latest tag would have to be updated as part of the release process.

Either would be fine with me and maybe contributors have other/better ideas for accomplishing the goal of routing first-time users to a stable release.

In any case, this would be more in line with the Apache Foundation's Release Policy:

Projects MUST direct outsiders towards official releases rather than raw source repositories, nightly builds, snapshots, release candidates, or any other similar packages.

(this came out of a Slack discussion)

[cwegener]

Just looking at what other projects do in regards to tagging on the Docker hub, there seems to be another tagname called stable. So, latest could possibly still be utilized for any developers who want to develop using docker images (not sure if that's an actual use case), and first-time users should be steered towards a new stable tag instead.

Edit Keeping the current meaning of the latest tag and introducing a new stable tag would also have the benefit that if anyone is currently depending on the previous behaviour of the latest tag, then they would not be broken by the tag name changes.

[sfirke]

Re: stable and lts tags, can docker image tags be manually modified on Docker Hub? Right now image tags are assigned at build-time, I believe in this file: https://github.com/apache/superset/blob/master/.github/workflows/docker_build_push.sh That works for our current tags, but I'm not sure how we'd tag an existing image as stable months after it has been released.

I'm open to that if it's possible, but I think the simplest next step is that when GITHUB_EVENT_NAME is "release", the images get tagged with official.

[cwegener]

@sfirke Once a docker image is built, you can use the docker image command to add more tags to the image. You can only upload tags to docker hub that are present on your local system. Docker hub can't make up tags for you (at least not that I know of)

[sfirke]

Interesting! I wonder how that would fit with the current model of builds being automatically tagged and pushed to Github during CI. Could the script have a new last step saying if GITHUB_EVENT_NAME = "official", then add a tag using docker image ...?

[cwegener]

I'm not really familiar with Github actions. Reading through the current Github actions folder doesn't really give me any clues about the current docker image publishing workflow. I can see that a container image gets built on every push to master.

In my mind, the simple logic here is that the continuously built container images get a tagged with 'devel' and 'latest'

And then there is another github action that only runs on the release branches (2.0, 2.1, etc) and also only when a git tag present. The 'newest' release branch that contains at least 1 git tag, is then the one that gets to use the 'stable' tag for tagging the container image with 'docker image' command before publishing the image.

[cwegener]

cc @sebastianliebscher

[sfirke]

While we are talking tags, I see that the images tagged latest / latest-dev appear to also be tagged master / master-dev. That seems redundant.

[sfirke]

Related issues:

• Clarify Superset Version Number in the UI #13674
• The docker image tagged latest does not deploy the latest stable version #16679
• Helm installation uses latest tag by default #17540

[rmskinsa2]

Just want to comment to say I support this idea.

[sfirke]

I have a related idea that would partly address this issue: #24987