-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
downloader: support for downloading bundles from an OCI registry #4558
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for contributing! 😃
I've got a couple of comments and questions inline.
We should ensure to
- add some docs about the new config option and what it does
- add a note to CHANGELOG.md about the incompatible changes to the
download
package
download/download.go
Outdated
@@ -75,7 +75,7 @@ type downloaderResponse struct { | |||
} | |||
|
|||
// New returns a new Downloader that can be started. | |||
func New(config Config, client rest.Client, path string) *Downloader { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[nit] Technically, this is backwards-incompatible. We'll have to call that out in the release notes.
updates <- &u | ||
}) | ||
*/ | ||
d := NewOCI(config, fixture.client, "ghcr.io/org/repo:latest", "/tmp/opa/").WithCallback(func(_ context.Context, u Update) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So this makes a call to the actual external registry? I'd rather avoid that in our tests... Can we leverage any test fixtures provided by the introduced dependencies, maybe?
Update: looks like that's what we're setting up in newTestFixture
, aren't we? Just want to be sure.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The tests do not make an actual call to a registry. It uses the fixture to run with the assets associated.
The commented code above allowed to create an actual client and test against a real OCI registry that I used while developing this. I will remove this commended block as it's no longer needed.
plugins/bundle/plugin.go
Outdated
@@ -385,6 +385,17 @@ func (p *Plugin) newDownloader(name string, source *Source) Loader { | |||
// wrap the callback to include the name of the bundle that was updated | |||
p.oneShot(ctx, name, u) | |||
} | |||
if client.Config().Type == "oci" { | |||
ociStorePath := "oci" // use current folder oci |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand this comment. Does it mean we're pulling things into the file system in folder oci/
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll update the comment here to clarify this a bit.
The OCI downloader needs a path for the local OCI store. If the PersistenceDirectory
is configured it adds an oci
folder in that path to store the downloaded files, but if the PersistenceDirectory
is nil it will add the oci
folder to the current directory.
I'm very open to suggestions here if we want to error out if the PersistenceDirectory
is not configured or set a different path.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if we want to error out if the PersistenceDirectory is not configured or set a different path.
I think this would be a decent choice. We'll need to ensure that the docs are clear, but if the OCI downloader needs a place to put stuff, we should require PersistenceDirectory to be set. Would it ever read back from that directory over service restarts...? Is it used as a download cache of some sort? 🤔
Since the dependencies are quite heavyweight on this, it would be nice if there were synergies. #3697 was blocked on its dependency-heaviness; if this was using the same golang libs, we'd open up the possibility to get cosign support in, too. By any chance, is this something you'd see possible here? From what I can tell, cosign does its heavy lifting wrt OCI via https://pkg.go.dev/github.com/google/go-containerregistry/pkg/v1. |
func (d *OCIDownload) pull(ref string) (*ocispec.Descriptor, []ocispec.Descriptor, error) { | ||
clientConfig := d.client.Config() | ||
authHeader := make(http.Header) | ||
resolver := docker.NewResolver(docker.ResolverOptions{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's a lot of stuff going on here... is it all about opa-as-client authenticating with different registries?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did a bit of refactoring in the latest commit here so it can be easier to read. I hope that clears things up.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some more questions, please bear with me!
plugins/bundle/plugin.go
Outdated
@@ -385,6 +385,17 @@ func (p *Plugin) newDownloader(name string, source *Source) Loader { | |||
// wrap the callback to include the name of the bundle that was updated | |||
p.oneShot(ctx, name, u) | |||
} | |||
if client.Config().Type == "oci" { | |||
ociStorePath := "oci" // use current folder oci |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if we want to error out if the PersistenceDirectory is not configured or set a different path.
I think this would be a decent choice. We'll need to ensure that the docs are clear, but if the OCI downloader needs a place to put stuff, we should require PersistenceDirectory to be set. Would it ever read back from that directory over service restarts...? Is it used as a download cache of some sort? 🤔
authz: | ||
service: ghcr-registry | ||
resource: ghcr.io/${ORGANIZATION}/${REPOSITORY}:${TAG} | ||
persist: true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could this also be persist: false
? I think I don't understand yet fully how persistence and OCI dependent on each other 😅
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can work with a persist false, but it needs a folder to store the layers it pulls from the OCI registry. I set that to the temporary folder if the persistence directory is not set.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would you mind describing that behaviour and the meaning of the configurable persist
when used with type: oci
in the docs somewhere?
Is this directory garbage-collected regularly, or would it be the user's responsibility to keep it from growing indefinitely? If so, it needs to be called out.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is no garbage-collection mechanism in place for this. I added a note in the changelog about the maintenance of the persistence folder.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is no garbage-collection mechanism in place for this. I added a note in the changelog about the maintenance of the persistence folder.
Uhm was that lost...? I can't find it 😅
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for bearing with me!
Here's another thing I'd appreciate a lot: some notes on how to debug this if stuff goes wrong. Say, there's an issue about the OCI feature breaking for a user; they've filed it on github; I'd like to reproduce it. What dummy OCI storage can I use? How do I push a policy to that OCI storage? What format is used? Can I "just" push a bundle.tar.gz?
I think most of this would make sense on the website, in an "OCI registry" sub-section somewhere. Right now, I wouldn't be able to answer those questions myself, I'm afraid 😅
authz: | ||
service: ghcr-registry | ||
resource: ghcr.io/${ORGANIZATION}/${REPOSITORY}:${TAG} | ||
persist: true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would you mind describing that behaviour and the meaning of the configurable persist
when used with type: oci
in the docs somewhere?
Is this directory garbage-collected regularly, or would it be the user's responsibility to keep it from growing indefinitely? If so, it needs to be called out.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's get this in 😅
To make this great, we'll need to have some OCI details on the web docs. The feature, cool as it may be, is still somewhat hidden right now.
docs/devel/OCI.md
Outdated
|
||
### OCI Downloader Debugging | ||
|
||
Before starting to run a step-by-step debug process on the OCI downloader you will need a policy image pushed to a repository. To do this at the moment the easiest method is to use the [policy CLI](https://github.com/opcr-io/policy) to build your image and push it to a repository. You can use the public [opcr-io](https://opcr.io/) repository, [GHCR](https://ghcr.io) or any other OCI compatible repository. As an alternative solution you can build the policy with the OPA CLI and using oras CLI to push it to a repository. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since you know exactly what to do and I don't -- could you please include an example of using the OPA CLI combined with the oras CLI? I think an example like that will help best to understand how this works in detail; short of diving into the opcr-io code itself.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ideally, that would be something in the docs, though. I'm afraid the docs added as part of this are quite minimal: they describe how to configure this, but they don't tell us how it works... if I know OPA well, and even if I know OCI, too, I still don't know how to put them together. I'd love to see something like "when stored in an OCI registry, a bundle is converted into an artifact of such and such form, with this metadata and these layers", perhaps with an example...? 🤔
If we want people to use OCI registries for managing their policies, we'll need an OCI subsection on the bundles docs, I suppose. I wouldn't expect people to look into the CHANGELOG, and much less into docs/deve/OCI.md 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added the note about the persist directory maintenance in the configuration.md doc
authz: | ||
service: ghcr-registry | ||
resource: ghcr.io/${ORGANIZATION}/${REPOSITORY}:${TAG} | ||
persist: true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is no garbage-collection mechanism in place for this. I added a note in the changelog about the maintenance of the persistence folder.
Uhm was that lost...? I can't find it 😅
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the contribution! Added few comments mainly around the download interface.
docs/content/configuration.md
Outdated
@@ -252,10 +252,33 @@ multiple services. | |||
| `services[_].tls.ca_cert` | `string` | No | The path to the root CA certificate. If not provided, this defaults to TLS using the host's root CA set. | | |||
| `services[_].tls.system_ca_required` | `bool` | No (default: `false`) | Require system certificate appended with root CA certificate. | | |||
| `services[_].allow_insecure_tls` | `bool` | No | Allow insecure TLS. | | |||
| `services[_].type` | `string` | No | Optional value that allows to use an "OCI" service type. This will allow bundle and discovery plugins to download bundles from an OCI registry. | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be helpful to state what the default is here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is an optional parameter, the default should be empty. You set it to "oci" when you want to use this downloader.
download/interface.go
Outdated
"github.com/open-policy-agent/opa/bundle" | ||
) | ||
|
||
type DownloaderInterface interface { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The downloader is an implementation of the Loader interface. So can we have another downloader implementation for OCI instead of a separate DownloaderInterface?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's a great point! I was about to comment (late) that Downloader
would be a better name for the interface (so we'd rename the struct), but this seems much better, @ashutosh-narkar
### OCI Downloader TODOs | ||
|
||
- Remove deprecated `ClearCache` function. | ||
- Implement appropriate `SetCache` function for the OCI downloader. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this mean we cannot have the bundle caching via Etag with OCI ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently the OCI spec documentation does not have any mentioning of using the Etag for caching. We can address this in a future PR.
download/oci_dowload.go
Outdated
close(done) | ||
} | ||
|
||
func (d *OCIDownload) loop(ctx context.Context) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There seems to be code in here that is similar to what we have for the http downloader. Can we move it in a common lib maybe ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suspect the OCI Downloader will diverge in the future. I think we might want to consider the possibility of having a lot less common code down the line.
I can't get this to work. I can't figure out how to push a bundle to ghcr.io. With both |
Assuming the ghcr somehow works, it still fails for me to download the bundle:
With this config:
|
I added a bit of information on how to push bundles to an OCI registry in OCI.md for now |
download/oci_dowload.go
Outdated
mtx sync.Mutex | ||
stopped bool | ||
persist bool | ||
longPollingEnabled bool |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How does long polling work with the OCI spec ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
According to the distribution specs at the moment there is no mention of pooling possibilities, but I left this in the OCI downloader as we might want to do some future improvements in the download section.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's remove that code then. It still lives in the HTTP downloader.
func (d *OCIDownloader) download(ctx context.Context, m metrics.Metrics) (*downloaderResponse, error) { | ||
d.logger.Debug("OCI - Download starting.") | ||
|
||
preferences := []string{fmt.Sprintf("modes=%v,%v", defaultBundleMode, deltaBundleMode)} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can delta bundles be used as well ? I guess as long as we have a tarball it should be supported , correct ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As @viovanov stated here #4518 (comment) this should work
@@ -5,6 +5,7 @@ go 1.16 | |||
require ( | |||
github.com/OneOfOne/xxhash v1.2.8 | |||
github.com/bytecodealliance/wasmtime-go v0.35.0 | |||
github.com/containerd/containerd v1.6.2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is going to pull down a lot of packages, isn't it ? Do we need to do this or is there some particular package / code segment that we are using ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some background: to keep OPA's dependencies light, we've resorted to "manual vendoring" of third party code. Like, we've copied over some jwt lib to back the jwt built-ins. If there is only a manageable amount of code needed from a third party dependency here, we could do the same. I haven't dug into this yet, so I don't know the answer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hrmm looking at this briefly, the oras packages have a pkg/auth/docker
, too. I'd expect that if we used that instead of "github.com/containerd/containerd/remotes/docker", we'd be able to cut down the dependencies a bit? WDYT @carabasdaniel?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We are using the docker resolver here as the source target for the oras.Copy method. At the moment I'm not sure we have a lighter solution for this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So my last clue here is probably a dead-end: https://github.com/oras-project/oras/blob/main/go.mod#L6 ORAS comes with the containerd/containerd module as a dependency, too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@carabasdaniel sorry I had missed your comment when I posted my last one -- here's a follow-up: #4558 (comment)
"sync" | ||
"time" | ||
|
||
"github.com/containerd/containerd/remotes/docker" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So the ORAS CLI manages to do the remotes setup using its own packages, alone -- I wonder if we could do that, too. Looking at the code in https://github.com/oras-project/oras/blob/d3eb505a33d23c552763e89357ea7dc9f8d44676/cmd/oras/resolver.go#L15, it's setting up the resolver by its own means; and looking at the login command, https://github.com/oras-project/oras/blob/d3eb505a33d23c552763e89357ea7dc9f8d44676/cmd/oras/login.go#L76, it also uses its own code for the client setup.
Can we do that, here, too? It would help us have one direct dependency less, after all. 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was mislead there, the resolver.go link shows that they do it just like us. Let's roll with this then.
Signed-off-by: carabasdaniel <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👏 Thank you for contributing this! It's quite a feat. I'll open a few follow-up issues with the known TODOs we want to address later.
🎈 🎉
# This is the 1st commit message: finalizing changes for formatting with sprintf Signed-off-by: Damien Burks <[email protected]> # This is the commit message open-policy-agent#2: updating changes to allow for multiple format strings Signed-off-by: Damien Burks <[email protected]> # This is the commit message open-policy-agent#3: fixing golint issues Signed-off-by: Damien Burks <[email protected]> # This is the commit message open-policy-agent#4: fixing golint issues Signed-off-by: Damien Burks <[email protected]> # This is the commit message open-policy-agent#5: making recommended change: package level variable Signed-off-by: Damien Burks <[email protected]> # This is the commit message open-policy-agent#6: adding support for explicit argument indexes Signed-off-by: Damien Burks <[email protected]> # This is the commit message open-policy-agent#7: format: don't add 'in' keyword import when 'every' is there (open-policy-agent#4607) Also ensure that added imports have a location set. Previously, `opa fmt` on the added test file would have panicked because the import hadn't had a location. Fixes open-policy-agent#4606. Signed-off-by: Stephan Renatus <[email protected]> # This is the commit message open-policy-agent#8: ast+topdown+planner: allow for mocking built-in functions via "with" (open-policy-agent#4540) With this change, we can replace calls to built-in functions via `with`. The replacement can either be a value -- which will be used as the return value for every call to the mocked built-in -- or a reference to a non-built-in function -- when the results need to depend on the call's arguments. Compiler, topdown, and planner have been adapted in this change. The included docs changes describe the replacement options further. Fixes first part of open-policy-agent#4449. (Missing are non-built-in functions as mock targets.) Signed-off-by: Stephan Renatus <[email protected]> # This is the commit message open-policy-agent#9: build(deps): bump google.golang.org/grpc from 1.45.0 to 1.46.0 (open-policy-agent#4617) # This is the commit message open-policy-agent#10: docs/policy-testing: use assignment operator in mocks (open-policy-agent#4618) Additionally, simplify one test example. Signed-off-by: Anders Eknert <[email protected]> # This is the commit message open-policy-agent#11: cmd/capabilities: expose capabilities through CLI (open-policy-agent#4588) There is a new command argument "capabilities". With this, it is possible to print the current capabilities version, show all capabilities versions & print any capabilities version, without the need of a file. Moreover, for the other commands which use the --capabilities flag, it is possible to give only the version number, without specifying a file. However, there are no breaking changes for those who use the capabilities file as an input for the flag. Unit tests were also written, in order to test the new argument and the changes made in ast. Fixes: open-policy-agent#4236 Signed-off-by: IoannisMatzaris <[email protected]> # This is the commit message open-policy-agent#12: format,eval: don't use source locations when formatting PE output (open-policy-agent#4611) * format: allow ignoreing source locations * cmd/eval: format disregarding source locations for partial result Before, we'd see this output: ``` $ opa eval -p -fsource 'time.clock(input.x)==time.clock(input.y)' # Query 1 time.clock(time.clock(input.x), input.y) ``` Now, we get the proper answer: `time.clock(input.y, time.clock(input.x))`. Note that it's a _display_ issue; the JSON output of PE has not been affected. Fixes open-policy-agent#4609. Signed-off-by: Stephan Renatus <[email protected]> # This is the commit message open-policy-agent#13: build(deps): bump github/codeql-action from 1 to 2 (open-policy-agent#4621) Bumps [github/codeql-action](https://github.com/github/codeql-action) from 1 to 2. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@v1...v2) --- updated-dependencies: - dependency-name: github/codeql-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> # This is the commit message open-policy-agent#14: status: Remove activeRevision label on all but one metric (open-policy-agent#4600) Having one activeRevision label on each of the prometheus metrics emitted by the status plugin has proven to be problematic with a large number of bundles. So with this change, 1. we keep the activeRevision label (just on) the last_success_bundle_activation metric. 2. the gauge gets reset, so we only keep the last active_revision (instead of keeping them all and therefore avoiding the situation where the /metrics output grows indefinitely) Fixes open-policy-agent#4584. Signed-off-by: cmuraru <[email protected]> # This is the commit message open-policy-agent#15: website: add playground button to navbar (open-policy-agent#4622) Addressing one tiny bit of open-policy-agent#4614. Signed-off-by: Stephan Renatus <[email protected]> # This is the commit message open-policy-agent#16: topdown/net: require prefix length for IPv6 in net.cidr_merge (open-policy-agent#4613) There are no default prefixes in IPv6, so if an IPv6 without a prefix is fed into net.cidr_merge, we'll return a non-halt error now. Before, we'd fail in various ways if a prefix-less IPv6 was fed into `net.cidr_merge`. With only one, we'd return `[ "<nil>" ]`, with two, we'd panic. Fixes open-policy-agent#4596. Signed-off-by: Stephan Renatus <[email protected]> # This is the commit message open-policy-agent#17: Dockerfile: add source annotation (open-policy-agent#4626) `org.opencontainers.image.source` URL to get source code for building the image (string) https://github.com/opencontainers/image-spec/blob/main/annotations.md Signed-off-by: Stephan Renatus <[email protected]> # This is the commit message open-policy-agent#18: build(deps): bump github.com/fsnotify/fsnotify v1.5.2 -> v1.5.4 (open-policy-agent#4628) https://github.com/fsnotify/fsnotify/releases/tag/v1.5.4 Signed-off-by: Stephan Renatus <[email protected]> # This is the commit message open-policy-agent#19: docs: update version in kubernetes examples (open-policy-agent#4627) Signed-off-by: yongen.pan <[email protected]> # This is the commit message open-policy-agent#20: bundle/status: Include bundle type in status information OPA has support for Delta Bundles. The status object already contains valuable information such as last activation timestamp but does not specify if the bundle was a canonical snapshot or delta. This change updates the bundle.Status object to include the bundle type string: either "snapshot" or "delta". This can be useful for status endpoints to differentiate between the bundle types. Issue: 4477 Signed-off-by: Bryan Fulton <[email protected]> # This is the commit message open-policy-agent#21: ast+topdown+planner: replacement of non-built-in functions via 'with' (open-policy-agent#4616) Follow-up to open-policy-agent#4540 We can now mock functions that are user-defined: package test f(_) = 1 { input.x = "x" } p = y { y := f(1) with f as 2 } ...following the same scoping rules as laid out for built-in mocks. The replacement can be a value (replacing all calls), or a built-in, or another non-built-in function. Also addresses bugs in the previous slice: * topdown/evalCall: account for empty rules result from indexer * topdown/eval: capture value replacement in PE could panic Note: in PE, we now drop 'with' for function mocks of any kind: These are always fully replaced in the saved support modules, so this should be OK. When keeping them, we'd also have to either copy the existing definitions into the support module; or create a function stub in it. Fixes open-policy-agent#4449. Signed-off-by: Stephan Renatus <[email protected]> # This is the commit message open-policy-agent#22: format: keep whitespaces for multiple indented same-line withs (open-policy-agent#4635) Fixes open-policy-agent#4634. Signed-off-by: Stephan Renatus <[email protected]> # This is the commit message open-policy-agent#23: downloader: support for downloading bundles from an OCI registry (open-policy-agent#4558) Initial support for open-policy-agent#4518. Configuration uses the 'services' config for registries, via the "type: oci" field. Bundles configured to pull from that service will then use OCI. ``` services: ghcr-registry: url: https://ghcr.io type: oci bundles: authz: service: ghcr-registry resource: ghcr.io/${ORGANIZATION}/${REPOSITORY}:${TAG} persist: true polling: min_delay_seconds: 60 max_delay_seconds: 120 persistence_directory: ${PERSISTENCE_PATH} ``` Service credentials are supported: if you want to pull from a private registry, use ``` services: ghcr-registry: url: https://ghcr.io type: oci credentials: bearer: token: ${GH_PAT} ``` If no `persistence_directory` is configured, the data is stored in a directory under /tmp. See docs/devel/OCI.md for manual steps to test this feature with some OCI registry (like ghcr.io). Signed-off-by: carabasdaniel <[email protected]> # This is the commit message open-policy-agent#24: Prepare v0.40.0 Release (open-policy-agent#4631) Signed-off-by: Stephan Renatus <[email protected]> # This is the commit message open-policy-agent#25: Prepare v0.41.0 development (open-policy-agent#4636) Signed-off-by: Stephan Renatus <[email protected]> # This is the commit message open-policy-agent#26: docs: Adding example for `rego.metadata.role()` usage (open-policy-agent#4640) Signed-off-by: Johan Fylling <[email protected]> # This is the commit message open-policy-agent#27: build(deps): bump oras.land/oras-go from 1.1.0 to 1.1.1 (open-policy-agent#4643) Bumps [oras.land/oras-go](https://github.com/oras-project/oras-go) from 1.1.0 to 1.1.1. - [Release notes](https://github.com/oras-project/oras-go/releases) - [Commits](oras-project/oras-go@v1.1.0...v1.1.1) --- updated-dependencies: - dependency-name: oras.land/oras-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> # This is the commit message open-policy-agent#28: build(deps): bump OpenTelemetry 1.6.3 -> 1.7.0 (open-policy-agent#4649) https://github.com/open-telemetry/opentelemetry-go/releases/tag/v1.7.0 https://github.com/open-telemetry/opentelemetry-go-contrib/releases/tag/v1.7.0 Signed-off-by: Stephan Renatus <[email protected]> # This is the commit message open-policy-agent#29: build(deps): bump github.com/containerd/containerd from 1.6.2 to 1.6.3 (open-policy-agent#4654) Bumps [github.com/containerd/containerd](https://github.com/containerd/containerd) from 1.6.2 to 1.6.3. - [Release notes](https://github.com/containerd/containerd/releases) - [Changelog](https://github.com/containerd/containerd/blob/main/RELEASES.md) - [Commits](containerd/containerd@v1.6.2...v1.6.3) --- updated-dependencies: - dependency-name: github.com/containerd/containerd dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> # This is the commit message open-policy-agent#30: Update k8s examples to the latest schema (open-policy-agent#4655) Signed-off-by: Víctor Martínez Bevià <[email protected]> # This is the commit message open-policy-agent#31: Fix incorrect padding claims (open-policy-agent#4657) Signed-off-by: Anders Eknert <[email protected]> # This is the commit message open-policy-agent#32: build(deps): bump github.com/containerd/containerd from 1.6.3 to 1.6.4 (open-policy-agent#4662) Bumps [github.com/containerd/containerd](https://github.com/containerd/containerd) from 1.6.3 to 1.6.4. - [Release notes](https://github.com/containerd/containerd/releases) - [Changelog](https://github.com/containerd/containerd/blob/main/RELEASES.md) - [Commits](containerd/containerd@v1.6.3...v1.6.4) --- updated-dependencies: - dependency-name: github.com/containerd/containerd dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> # This is the commit message open-policy-agent#33: build(deps): bump docker/setup-qemu-action from 1 to 2 (open-policy-agent#4668) # This is the commit message open-policy-agent#34: build(deps): bump docker/setup-buildx-action from 1 to 2 (open-policy-agent#4669) Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 1 to 2. - [Release notes](https://github.com/docker/setup-buildx-action/releases) - [Commits](docker/setup-buildx-action@v1...v2) --- updated-dependencies: - dependency-name: docker/setup-buildx-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> # This is the commit message open-policy-agent#35: build(deps): github.com/bytecodealliance/wasmtime-go 0.35.0 -> 0.36.0 (open-policy-agent#4652) * build(deps): bump wasmtime-go: 0.35.0 -> 0.36.0 * internal/wasm: adapt to using epoch-based interruption Looks like we don't get frames for this. Also, there is currentlty no better way than comparing the message, as the trap code isn't surfaced (yet). Fixes open-policy-agent#4663. Signed-off-by: Stephan Renatus <[email protected]> # This is the commit message open-policy-agent#36: ecosystem: Add Sansshell (open-policy-agent#4674) Signed-off-by: James Chacon <[email protected]> # This is the commit message open-policy-agent#37: topdown: Add units.parse builtin (open-policy-agent#4676) This function works on all base decimal and binary SI units of the set: m, K/Ki, M/Mi, G/Gi, T/Ti, P/Pi, and E/Ei Note: Unlike `units.parse_bytes`, this function is case sensitive. Fixes open-policy-agent#1802. Signed-off-by: Philip Conrad <[email protected]> # This is the commit message open-policy-agent#38: docs/contrib-code: Add capabilities step to built-in functions tutorial (open-policy-agent#4677) Signed-off-by: Philip Conrad <[email protected]>
Initial PR for #4518
This PR contains the initial changes for the downloader to support downloading bundles from an OCI registry.