Caching and atomicity of attestation storage

[laurentsimon]

attestation may be stored in an attestation store or an existing registry. However, it's possible users make mistake and delete or alter the attestation. An existing deployment would then start failing, eg when scaling out during high traffic. What we need is atomicity of the deployment, ie the image is either _always passing verification or always failing verification.

Ideally the attestation should be part of the k8 config file, eg as an annotation for the pods. This annotation could be created by a gitops tool like ArgoCD. we could also do that ourselves:

1. We verify using an annotation (cache 1). If not present, we look at the config map (cache 2). if not present, we retrieve from the attestation store.
2. We will the cache

Cache 1 has the lifetime of a pod. Cache 2 has lifetime of a cluster. namespace. So if a pod is killed and restarted as-is, we'd still get the cache 2 hit.