Ensure security vulnerabilities are patched in MCR images

[peterbom]

This includes changes for:

1. A scheduled pipeline to scan and patch the latest published images in GHCR and ACR/MCR.
2. Extra steps in our MCR publishing pipeline to patch before publishing.
3. Merging of the GHCR and MCR publishing pipelines to ensure consistency between container registries, and because the build/scan/patch process requires images to be published to a public registry.

Example runs:

• Build and Publish
• Patch

Notes on why we're making these particular changes:

Should we create a pipeline that runs on a schedule, or on demand? I think it makes more sense to run it on a schedule. Reason: any issues that we detect should be in our dependency tree, and will change when new vulnerabilities are discovered in those dependencies. We won't know when that happens, so nothing will prompt us to run the pipeline.

Should that pipeline scan the image built from the latest code in master, or the latest published images? The published images are the ones that actually matter, because they're being used, but it would still be useful to know if our code introduced a dependency with a vulnerability. For a pipeline that runs on a schedule, I think we should be checking the latest published images, because our code doesn't change on a regular schedule. (In theory we could patch older images too, but that would be more work and massively increase pipeline execution times, and we can add that in later if we really need it.)

Should the pipeline just create a report and send some notification if there are vulnerabilities, and let us know so we can manually run a pipeline to patch the images? Or just go ahead and patch them? I think it should go ahead and patch. Realistically, we're not going to analyze the report ourselves and have the knowledge to make good decisions. And even if there's a risk that the patch breaks our functionality, security is more important than functionality. We are keeping unpatched images available with the 'unpatched tag suffix, just in case something gets broken.

Can we test the patched images before publishing? We have no automated tests for images, and I don't think it'd be trivial to create them. But if we add the patching to the GHCR publishing pipeline, we can manually check the published images from VS Code, which we should probably be doing before publishing to MCR anyway.

[codecov-commenter]

Codecov Report

Merging #233 (3cdc0b4) into master (a73086c) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #233   +/-   ##
=======================================
  Coverage   80.53%   80.53%           
=======================================
  Files          14       14           
  Lines         786      786           
=======================================
  Hits          633      633           
  Misses         94       94           
  Partials       59       59           

[peterbom]

Note: need to upgrade to 1.0.1 which adds a curl retry when getting the latest copa version.

[peterbom]

Updated, but now waiting for the next release of copa which will include project-copacetic/copacetic#377