v0.29.4

🚨 Upgrade Notice: Use opa_linux_amd64_static instead of opa_linux_amd64 🚨

What changed and why?

The v0.29 release changes the pre-built opa_linux_amd64 binary to better support wasm-based policy execution. The change requires glibc 2.18+ as well as libgcc.

Who is affected by this change?

This change affects users running alpine-based Docker images, CentOS 7, etc.

What should affected users do?

If you currently deploy the pre-built OPA binaries to one of these systems, update your automation to download opa_linux_amd64_static instead of opa_linux_amd64. Going forward, the opa_linux_amd64_static binary is recommended for systems that do not have the required system libraries.

If you currently build OPA from source and deploy to one of these systems, update your build to produce a statically linked executable by setting the CGO_ENABLED and WASM_ENABLED flags (e.g., make build WASM_ENABLED=0 CGO_ENABLED=0).

Related issues

#3499
#3532
#3528

Miscellaneous

• bundle: Implement a DirectoryLoader for fs.FS (#3493) (#3489) authored by @simongottschlag

v0.29.3

This bugfix release addresses another edge case in function evaluation (#3505).

v0.29.2

This is a bugfix release to resolve an issue in topdown's function output caching (#3501)

v0.29.1

This release contains a number of enhancements and fixes.

SDK

• This release includes a new top-level package to support OPA integrations in Go programs: github.com/open-policy-agent/opa/sdk. Users that want to integrate OPA as a library in Go and expose features like bundles and decision logging should use this package. The package is controlled by specifying an OPA configuration file. Hot reloading is supported out-of-the-box. See the GoDoc for the package docs for more details.

Server

• A deadlock in the bundle plugin during shutdown has been resolved (#3363)
• An issue between bundle signing and bundle persistence when multiple data.json files are included in the bundle has been resolved (#3472)
• The github.com/open-policy-agent/opa/runtime#Params struct now supports a router parameter to enable custom routes on the HTTP server.
• The bundle manifest can now include an extra metadata key where arbitrary key-value pairs can be stored. Authored by @viovanov
• The bundle plugin now supports file:// urls in the resource field for test purposes.
• The decision log plugin emits a clearer message at DEBUG instead of INFO when there is no work to do. Authored by andrewbanchich
• The discovery plugin now supports a resource configuration field like the bundle plugin. Similarly, the resource is treated as the canonical setting to identify the discovery bundle.

Tooling

• The opa test timeout as been increased to 30 seconds when benchmarking (#3107)
• The opa eval --schema flag has been fixed to correctly set the schema when a single schema file is passed
• The opa build --debug flag output has been improved for readability
• The array.items JSON schema value is now supported by the type checker
• The opa fmt subcommand can now exit with a non-zero status when a diff is detected (by passing --fail)
• The opa test subcommand no longer emits bogus file paths when fed a file:// url

Built-in Functions

• The http.send built-in function falls back to the system certificate pool when the tls_ca_cert or tls_ca_cert_env_variable options are not specified (#2271) authored by @olamiko

Evaluation

• The order of support rules emitted by partial evaluation is now deterministic (#3453) authored by @andrehaland
• The big number performance regression caught by the fuzzer has been resolved (#3262)
• The evaluator has been updated to memoize calls to rules with arguments (functions) within a single query. This avoids recomputing function results when the same input is passed multiple times (similar to how complete rules are memoized.)

WebAssembly

• The wasm target no longer panics if the OPA binary does not include a wasm runtime (#3264)
• The interrupt handling mechanism has been rewritten to make safe use of the wasmtime package. The SDK also returns structured errors now that are more aligned with topdown. (#3225)
• The SDK provides the subset of required imports now (which is useful for debugging with opa_println in the runtime library if needed.)
• The opa_number_float type has been removed from the value library (it was unused after moving to libmpdec)
• The runtime library builder has been updated to use llvm-12 and the wasmtime-go package has been updated to v0.27.0

Documentation

• The HTTP API authorization tutorial has been updated to show how to distribute policies using bundles
• The Envoy tutorial has been tweaked to show better path matching examples

Infrastructure

• The release-patch script has been improved to deal with this file in bugfix/patch releases (#2533) authored by @jjshanks
• The Makefile check targets now rely on golangci-lint and many linting errors have been resolved (authored by @willbeason)
• Multiple nightly fuzzing and data race issues in test cases have been resolved

v0.28.0

This release includes a number of features, enhancements, and fixes. The default
branch for the Git repository has also been updated to main.

Schema Annotations

This release adds support for annotations. Annotations allow users to declare metadata on rules and packages. Currently, OPA supports one form of metadata: schema declarations. For example:

package example

# METADATA
# schemas:
# - input: schema.service
deny["service is missing required 'owner' label"] {
  input.kind == "Service"
  not input.metadata.labels.owner
}

# METADATA
# schemas:
# - input: schema.deployment
deny["deployment replica count too low for 'production' namespace"] {
  input.kind == "Deployment"
  input.metadata.namespace == "production"
  object.get(input.spec, "replicas", 1) < 3
}

Users can include schema annotations in their policies to tell OPA about the structure of external data loaded under input or data. By learning the schema of base documents, OPA can surface mistakes in the policy at authoring time (e.g., referring to a non-existent field in a JSON object or calling a built-in function with an invalid value.) For more information on the annotations and schema support see the Type Checking page in the documentation. In the future, annotations will be expanded to support other kinds of metadata and additional tooling will be added to leverage them.

Server

• The server now automatically sets GOMAXPROCS when running inside of a container that has cgroups applied. This helps the Go runtime avoid consuming too many CPU resources and being throttled by the kernel. (#3328)
• The server now logs an error if users enable the token authentication mode without a corresponding authorization policy. (#3380) authored by @kale-amruta
• The server now supports a GET /v1/config endpoint that returns OPA's active configuration. This API is useful if you need to debug the running configuration in an OPA configured via Discovery. (#2020)
• The server now respects the ?pretty option in the v0 API (#3332) authored by @clarshad
• The Bundle plugin is more forgiving when it comes to Etag processing on HTTP 304 responses (#3361)
• The Decision Log plugin now supports a "Decision Per Second" rate limit configuration setting.
• The Status plugin can now be configured to use a custom reporter similar to the Decision Log plugin (e.g., so that Status messages can be sent to AWS Kinesis, etc.)
• The Status plugin now reports the number of decision logs that are dropped due to buffer limits.
• The service clients can authenticate with the Azure Identity OAuth2 implementation the client credentials JWT flow is used (#3372)
• Library users can now customize the logger used by the plugins by providing the plugins.Logger option when creating the plugin manager.

Tooling

• The various OPA subcommands that accept schema files now accept a directory tree of schemas instead of only a single schema.
• The opa refactor move subcommand was added to support package renaming use cases (#3290)
• The opa check subcommand now supports a -s/--schema flag like the opa eval subcommand.

Documentation

• The Management API docs have been restructured so that each API has a dedicated page. In addition, the Bundle API docs now include getting started steps for cloud-provider specific services (e.g., AWS, GCP, Azure, etc.)

Security

• OPA now supports PKCS8 encoded EC private keys for JWT verification (which includes service authentication, bundle verification, and verification built-in functions) (#3283). Authored by @andrehaland.
• The bundle signing and verification APIs have been updated to support custom signers/verififers (#3336). Authored by @gshively11.

Evaluation

• The time.diff function was added to support calculating differences between date/time values (#3348) authored by @andrehaland
• The units.parse_bytes function now supports floating-point values (#3297) authored by @andy-paine
• The evaluator was fixed to use correct bindings when evaluating the full-extent of a partial rule set. This issue was causing unexpected undefined results and evaluation errors in some rare cases. (#3369 #3376)
• The evaluator was fixed to correctly generate package paths when namespacing is disabled partial evaluation. (#3302).
• The http.send function no longer errors out on invalid Expires headers. (#3284)
• The inter-query cache now serializes elements on insertion thereby reducing memory usage significantly (because deserialized elements carry a ~20x cost.) (#3042)
• The rule indexer was fixed to correctly handle mapped and non-mapped values which could occur with glob.match usage (#3293)

WebAssembly

• The opa eval subcommand now correctly returns the set of all variable bindings and expression values when the wasm target is enabled. Previously it returned only set of variable bindings. (#3281)
• The glob.match function now handles the default delimiter correctly. (#3294)
• The opa build subcommand no longer requires a capabilities file when the wasm target is enabled. If capabilities are not provided, OPA will use the capabilities for its own version. (#3270)
• The opa build subcommand now dumps the IR emitted by the planner when --debug is specified.
• The opa eval subcommand no longer panics when a policy fails to type check and the wasm target is enabled.
• The comparison functions can now return false instead of either being true or undefined. (#3271)
• The internal wasm runtime will now correctly return CancelErr to indicate cancellation errors (instead of BuiltinErr which it returned previously.)
• The internal wasm runtime now correctly handles non-halt built-in errors (#3320)
• The planner no longer generates unexpected scan statements when negation used over base documents under data (#3279) and (#3305)
• The planner now correctly discards out-of-scope variables when exiting comprehensions (#3325)
• The rego package no longer panics when the wasm target is enabled and undefined functions are encountered (#3251)
• 🎈 The remaining exceptions in the e2e test framework for the internal wasm runtime have been resolved.

Build

• The make image target now uses the CI image for building the Go binary. This avoids platform-specific build issues by building the Go binary inside of Docker.

v0.27.1

This release contains a fix for crashes experienced when configuring OPA to use S3 signing as service credentials (#3255).

In addition to that, we have a small number of enhancements and fixes:

Tooling

• The eval subcommand now allows using --import without using --package. Authored by @onelittlenightmusic, #3240.

Compiler

• The ast package now exports another method for JSON conversion, ast.JSONWithOpts, that allows further options to be set (#3244.

Server

• REST plugins using s3_signing as credentials method can now include the specified service in the signature (SigV4). Authored by @cogwirrel, #3210.

Documentation

• Remove soon-to-be deprecated any and all from the Policy Reference (#3241) -- see also #2437.
• Add missing discovery.service field to Discovery configuration table (#3237).
• Fix dead links to the Envoy pages (#3248).

WebAssembly

• Executions using the internal Wasm SDK will now be interrupted when the provided context is done (cancelled or deadline reached).
• The generated Wasm modules could become much smaller: unused functions are replaced by unreachable stubs, and the heavyweight runtime components related to regular expressions are excluded when none of the regex-related builtins are used: glob.match, regex.is_valid, regex.match, regex.is_valid, and regex.find_all_string_submatch_n.
• The Wasm runtime now allows passing in the time to be used for evaluation, enabling callers to control the time-of-day observed by Wasm compiled policies.
• Wasmtime runtime has been updated to the latest version (v0.24.0).

v0.27.0

This release contains a number of enhancements and bug fixes.

Tooling

• The eval subcommand now supports a -s/--schema flag that accepts a JSON schema for the input document. The schema is used when type checking the policy so that invalid references to (or operations on) input data are caught at compile time. In the future, the schema support will be expanded to accept multiple schemas and rule-level annotations. See the new Schemas documentation for details. Authored by @aavarghese and @vazirim.
• The eval, test, bench and REPL subcommands now supports a -t/--target flag to set the evaluation engine to use. The default engine is rego referring to the standard Rego interpreter in OPA. Users can now select wasm to enable Wasm compilation and execution of policies (#2878).
• The eval subcommand now supports a raw option for -f/--format that is useful in bash scripts. Authored by @jaspervdj-luminal.
• The test framework now supports "skippable" tests. Prefix the test name with todo_ to have the test runner skip the test, e.g., todo_test_allow { ... }.
• The eval subcommand now correctly supports the --ignore flag. Previously the flag was not being applied.

Server

• The POST /v1/compile API now supports a ?metrics query parameter similar to other APIs. Authored by @jkbschmid.
• The directory used for persisting downloaded bundles can now be configured. See the Configuration page for details.
• The HTTP Decision Logger plugin no longer blocks server shutdown for the grace period when there are no logs to upload.
• The Bundle plugin now unregisters listeners correctly. This issue would cause listeners to be invoked when bundle updates were dispatched even if the listener was unregistered (#3190).
• The server now correctly decodes policy IDs in the HTTP request URL. Authored by @mattmahn (#2116).
• The server now configures the http_request_duration_seconds metric (for all of the server endpoitns) with smaller, more granular buckets that better map to actual response latencies from OPA. Authored by @luong-komorebi (#3196).

Security

• PKCS8 keys are now supported when signing bundles and communicating with control plane services. Previously only PKCS1 keys were supported (#3116).
• The built-in OPA HTTP API authorizer policy can now return a reason to explain why a request to the OPA API is denied (#3056). See the Security documentation for details. Thanks to @ajanthan for helping improve this.

Compiler

• The compiler can be configured to emit debug messages that explain comprehension indexing decisions. Debug messages can be enabled when running opa build with --debug.
• A panic was fixed in one of the rewriting stages when comprehensions were used as object keys (#2915)

Evaluation

• A bug in big integer comparison was fixed. This issue was discovered when comparing serial numbers from X.509 certificates. Authored by @andrehaland (#3147).
• The io.jwt.decode_verify function now uses the environment supplied time-of-day value instead of calling time.Now() (#3105).

Documentation

• The documentation now includes a dedicated section the OPA-Envoy integration. See https://www.openpolicyagent.org/docs/latest/envoy-introduction/ for details.
• The ecosystem page now ranks integrations by number of unique domains instead of the sheer number of references.

WebAssembly

• The data document no longer needs to be initialized to an empty object (#3130).
• The mpd library is now initalized by the module's Start function (#3110).
• The planner now longer re-plans rules blindly when with statements are encountered (#3150).
• The planner and compiler now support dynamic dispatch. Previously the planner would enumerate all functions and invocation was controlled at runtime (#2936).
• The compiler now inserts memoization instructions into function bodies instead of at callsites. This reduces the number of wasm instructions in the resulting binary (#3169).
• The wasmtime runtime is now the default runtime used by OPA to execute compiled policies. The new runtime no longer leaks memory when policies are reloaded.
• The planner and compiler now intern strings and booleans and implement a few micro-optimizations to reduce the size of the resulting binary.
• The capabilities support has been updated to include an ABI major and minor version for tracking backwards compatibility on compiled policies (#3120).

Backwards Compatibility

• The opa test subcommand previously supported a -t flag as shorthand for --timeout. With this release, the -t shorthand has been redefined for --target. After searching GitHub for examples of opa test -t (and finding nothing) we felt comfortable making this backwards incompatible change.
• The Go version used to build the OPA release has been updated from 1.14.9 to 1.15.8. Because of this, TLS certificates that rely on Common Name for verification are no longer supported and will not work. For more information see golang/go#39568.

v0.26.0

This release contains a number of enhancements and bug fixes.

Built-in Functions

• This release includes a number of built-in function improvements for Wasm compiled policies. The following built-in functions have been implemented natively and no longer need to be supplied by SDKs: graph.reachable, json.filter, json.remove, object.get, object.remove, and object.union.

• This release fixes several bugs in the Wasm implementation of certain regex built-in functions (#2962), format_int (#2923) and round (#2999).

• This release adds ceil and floor built-in functions. Previously these could be implemented in Rego using round however these are more convenient.

Enhancements

• OPA has been extended support OAuth2 JWT Bearer Grant Type and OAuth2 Client Credential JWT authentication options for communicating with control plane services. This change allows OPA to use services that rely on Ping Identity as well as GCP service accounts for authentication. OPA has also been extended to support custom authentication plugins (thanks @gshively11).

• OPA plugins can now enter a "WARN" state to indicate they are operating in a degraded capacity (thanks @gshively11).

• The opa bench command can now benchmark partial evaluation queries. The options to enable partial evaluation are shared with opa eval. See opa bench --help for details.

• Wasm compiled policies now contain source locations that are included inside of runtime error messages (such as object key conflicts.) In addition, Wasm compiled policies only export the minimal set of APIs described on the WebAssembly#exports page.

Fixes

• ast: Fix parsing of numbers to reject leading zeroes (#2947) authored by @LCartwright.
• bundle: Fix loader to only verify bundle keys if configured to do so (#3028).
• cmd: Fix build to avoid packaging policy.wasm twice (#3007).
• cmd: Fix pretty-printed PE output to hide spurious blank lines
• server: Fix false-positive in bundle root check that would prevent data updates in some cases (#2868).
• server: Fix query cache to respect ?instrument option (#3000).
• server: Fix server to support discovery on inter-query cache configuration
• topdown: Fix PE to avoid generating expressions that do not type check (#3012).
• wasm: Fix planner to avoid generating a conflict error in some cases (#2926).
• wasm: Fix planner to generate correct virtual document iteration instructions (#3065).
• wasm, topdown: Fix with keyword handle to ensure last statement wins (#3010).
• wasm: Fix planner to handle assignment conflicts correctly when else keyword is used (#3031).

Documentation

• Add new section on integrating policies with OAuth2 and OIDC.
• Update Kubernetes admission control tutorial to work as non-root user.
• Fix link to signing documentation (#3027) authored by @princespaghetti.

Backwards Compatibility

• Previously, OPA deduplicated sets and objects in all cases except when iterating over/referring directly to values generated by partial rules. This inconsistency would only be noticed when running ad-hoc queries or within policies when aggregating the results of array comprehensions (e.g., count([1 | p[x]]) could observe duplicates in p.) This release removes the inconsistency by deduplicating sets and objects in all cases (#429). This was the second oldest open issue on the project.

Deprecations

• OPA now logs warnings when it receives legacy bundle config sections instead of the bundles section introduced in v0.13.0.

v0.25.2

This release extends the HTTP server authorizer (--authorization=basic) to supply the HTTP message body in the input document. See the Authentication and Authorization section in the security documentation for details.

v0.25.1

This release contains a fix for running OPA under Docker with a non-default working directory (#2974):

/opa: error while loading shared libraries: libwasmer.so: cannot open shared object file: No such file or directory