[Spike] [60min *max*] Investigate binderhub's image re-building logic

[consideRatio]

We have a bug making us rebuild images already built in binderhub when working against quay.io as reported in #4315. Its hard to say what to do in order to resolve that without doing an investigation and learning a bit about how things work.

• investigate binderhub's image re-building logic
  • how and where does it decide if should re-build an image?
  • is there authentication steps involved in this logic, and does it accept configuration related to it?

[consideRatio]

how and where does it decide if should re-build an image?

Answer: If get_image_manifest returned truthy

is there authentication steps involved in this logic, and does it accept configuration related to it?

Yes. There is DockerRegistry.username DockerRegistry.password config, also DockerRegistry.url and DockerRegistry.token_url. These configuration options also have default values that stem from inspecting a docker config, and its also possible to provide a docker config in order to provide credentials.

Me and @sgibson91 tried to configure this via binderhub-service.config.DockerRegistry in the opensci small binder chart, but didn't get it functional all the way. We probably need to declare url to the registry as well.

Another alternative approach is to provide the binderhub pod a docker config with credentials, just like we do for the builder pods. There isn't any extraVolumes and extraVolumeMounts currently, so this would have to be done by a change in binderhub-service one way or another no matter what.