Docker Compose file(s)

[mreferre]

Hi there! I'd like to start a (philosophical) discussion about how to bring up Apache Superset onto ECS/Fargate using the existing Docker Compose file(s). For background, Docker has introduced a new feature that allows Docker customers to deploy their compose files to Amazon ECS in addition to bring up the stack locally. Here is a blog I wrote that talks about this capability.

I am not asking this community to go deep into the Compose/ECS integration... I am more keen to understand the layout of the compose files in this repo.

Onto my doubts.... ( I will disclose I don't know Superset at all so bear with me please). This repo contains a docker-compose.yml and a docker-compose-non-dev.yml file(s). They are slightly different but the one thing that drives me off is that the non-dev file (should I interpret it as "prod"?) does a bind mount like ./docker:/app/docker . Of course anything like this would not work with ECS Fargate because the ./docker folder does not exist locally on the Fargate task and it only exist on the system that cloned this repo.

It's as if to deploy in production it is assumed to clone the repo on a docker host and compose up the "non-dev" compose file?

I did work this around by 1) creating a new image with a new Dockerfile that does FROM the original image (apache/superset:latest-dev) and then ADD the docker folder (basically simulating the bind mount that happens in the compose). Now if you take the docker-compose-non-dev.ymland you comment out the bind mount (#- ./docker:/app/docker) and you change the image name to the new image you created above (and that you pushed somewhere where ECS/Fargate can pull) your stack will come up just fine in ECS/Fargate (with a docker compose up in the ecs context).

The specific questions I have are:

• Are the compose file (docker-compose-non-dev.yml) supposed to be used to deploy "in production"
• is it expected that a "production" deployment is done by "upping" the compose above on a Docker host that has cloned the repo?
• why is the docker-compose-non-dev.yml referencing a dev image (apache/superset:latest-dev)?

I may have more questions depending on where this goes.

Thanks for your patient!

[srinify]

Hi @mreferre generally speaking the Docker Compose setup in Superset is really only for one-command spin-up locally. This is convenient for people who know just enough about the command line but may not be full stack software engineers.

For a while, there was only docker-compose.yml which built frontend assets from scratch. This meant that many many people would run docker-compose up and see "a blank screen". The frontend assets weren't done building yet, and this was confusing to newcomers.

So @nytai created docker-compose-non-dev.yml to just use pre-built frontend assets. This makes it less ideal for development of course, hence the deliberate naming of non-dev :]