Prepare for using azure cleanroom in tests

[DomAyre]

Motivation

When attempting to add azure cleanroom support, I noticed that the KMS code defines a "workspace" which is assumed to have the same structure as the workspace created when running sandbox.sh from ghcr.io/microsoft/ccf/app/dev/${CCF_PLATFORM}:${TAG}.

Since the network created by azure cleanroom doesn't have this structure we need to update the code that has this assumption.

Changes

• Simplify the devcontainer
• Have self contained script directories for different backends
  • up.sh script sets env variables such as KMS_URL which are specific to type of backend
• Don't assume anything about CCF workspace to enable using Azure Cleanroom
  • In practice this means setting full paths to things like member and service certs instead of setting a workspace and assuming the structure
• Added scripts for calling each endpoint
  • Makes local testing much easier
• Update makefile targets to use this new code

[beejones]

These scripts force the usage of containers. So running KMS locally will no longer be possible. By the way, this is how I do my dev work. What are the limitations going forward?
In development you start frequently KMS. Containers have this persistent state behavior of you don't completely remove them. Meaning rebuilding of the container for every dev step.

[DomAyre]

Using CCF only from the container is indeed a choice we need to make carefully, my reasoning for supporting this choice are:

• It mimics how the real system will behave (i.e. Run CCF network somewhere else, and apply KMS code/policies/etc to it)
• It allows us to have a much simpler dev environment as you don't need to work on top of the CCF dev container or install CCF manually
• It makes running against KMS on different styles of CCF network trivial (sandbox locally, sandbox in aci, mCCF while it lasts, azure-cleanroom etc.)

I believe the considerations you raise aren't bad enough to outweigh the benefits for the following reasons:

• Persistent state: Using docker compose down cleans up for you, I've never had trouble with it
• Rebuilding:
  • Since the container image is just the official CCF dev image (i.e. no KMS code), it doesn't typically change while developing the KMS
  • In my experience starting or recreating the container takes ~5 seconds, building the kms code often takes longer
  • If you don't mind keeping the current KV/ledger of a running KMS this scheme even lets you update the code without stopping or recreating the CCF network

[KenGordon]

How much work would it be to support Ronny's way of working as well?

[DomAyre]

It's possible, we would need:

• A local environment with virtual CCF installed and all it's dependencies
  • See the changes to devcontainer.json, requires developing inside the ccf/dev container or complex setup
• We can then add /scripts/ccf/sandbox-local-non-container/ directory with up and down scripts
  • We would need a some extra code to handle the lifecycle stuff which docker compose does for us in this PR

[DomAyre]

Personally I see that as a pain, and I don't see any benefits vs just starting the off the shelf CCF container and treating it like every other use case of CCF