All packages, except for `cmd` and `internal`, have been moved into a new `v1` root package.
Old packages are kept for backwards-compatibility reasons. All contained code is replaced with simple type aliases and proxy functions to `v1` implementations.
Old packages default to the Rego v0 syntax, new `v1` packages default to the Rego v1 syntax.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* Respect runtime rego-version in RESTful policy API
Updating `/v1/policies` RESTful server endpoint to respect the `--v0-compatible`/`--v1-compatible` flags.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
The previous version has been failing without any good reason for me,
so let's try this.
About the version pick: It's not the latest version (v1.62.0 at the
moment), because that would introduce a new revive rule,
redeclares-builtin-id, and that flags every variable called `min` or
`max` in the code base. I had started addressing these, but they were
just too many.
The new issues related to this version are mostly that it complains
whenever it finds a non-static string that makes its way into a printf-
like function. However, that's a common pattern in some place here, so
I've sprinkled some nolint:govet on it.
Signed-off-by: Stephan Renatus <stephan@styra.com>
A new optimized read mode has been added to the default in-memory store, where data written to the store is eagerly converted to AST values (the data format used during evaluation). This pre-converted data is faster to read, and won’t cause memory spikes during load; but comes with slower data writes (affects startup and bundle load/update time) and a larger lowest overall memory footprint for OPA. Can be enabled for `opa run`, `opa eval`, and `opa bench` by setting the `—optimize-store-for-read-speed`. See http://localhost:8888/docs/edge/policy-performance/#storage-optimization.
Implements: #4147
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Co-authored-by: Ashutosh Narkar <anarkar4387@gmail.com>
This is the last few tests to be refactored before all tests are compatible with the v1-by-default switch coming in OPA 1.0.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
To make OPA behave as v0.x post v1.0 release.
If used simultaneously with `--v1-compatible` flag, the `--v0-compatible` flag takes precedence.
Also, future-proofing `cmd` package tests for 1.0.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
In addition to the vendor update, a test case was updated to account for a
prometheus change where the Content-Encoding header is unset if data is uncompressed.
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
Adding a resource map to the distributed_tracing
config. Entries in this map will be passed through to the OpenTelemetry SDK where they will be
added as resource attributes. The available resource attributes are service.namespace,
service.version and service.instance.id. see
https://opentelemetry.io/docs/specs/semconv/resource/Fixes: #6492
Signed-off-by: Brett McBride <brett@deakin.edu.au>
Previously the HTTP request context was included in the
request context at the info access log level. This means
if the access log level was set to error for instance,
the context would not include the http request context and
hence it would not be surfaced via decision logs.
This change always add the http request context to the request
context so that HTTP info like headers will be surfaced via decision
logs irrespective of the access log level.
Fixes: #6951
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
This change updates the repl so that the modules in the
provided bundle are parsed based on the `rego_version` attribute
in the bundle manifest. Currently that is ignored which leads
to parsing failures.
Fixes: #6872
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
It would be useful if users had the ability to enhance the
decision log with info from the incoming HTTP request such as
headers. This change allows users to configure headers whose
values if present in the incoming HTTP request would be
surfaced via the decision log. This can be extended in the
future to include more context from the request.
Fixes: #6693
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
When running OPA with the distributed tracing option enabled,
the OpenTelemetry trace exporter is not gracefully shut down
when the server is stopped.
This PR fixes that issues by moving the trace exporter shutdown
in the gracefulServerShutdown function.
Fixes: #6651
Signed-off-by: Nicolas Chotard <nicolas.chotard@backmarket.com>
Signed-off-by: David Wobrock <david.wobrock@backmarket.com>
Previously if Discovery was enabled, other features like bundle downloading and status reporting could not be configured manually.
The reason for this was to prevent OPAs being deployed that could not be controlled through discovery. It's possible that
the system serving the discovered config is unaware of all options locally available in OPA. Hence, we relax the configuration
check when discovery is enabled so that the bootstrap configuration can contain plugin configurations. In case of conflicts,
the bootstrap configuration for plugins wins. These local configuration overrides from the bootstrap configuration are included
in the Status API messages so that management systems can get visibility into the local overrides.
**In general, the bootstrap configuration overrides the discovered configuration.** Previously this was not the case for all
configuration fields. For example, if the discovered configuration changes the `labels` section, only labels that are
additional compared to the bootstrap configuration are used, all other changes are ignored. This implies labels in the
bootstrap configuration override those in the discovered configuration. But for fields such as `default_decision`, `default_authorization_decision`,
`nd_builtin_cache`, the discovered configuration would override the bootstrap configuration. Now the behavior is more consistent
for the entire configuration and helps to avoid accidental configuration errors.
Fixes: #5722
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
Adding a global `rego_version` attribute to bundle manifest, to inform OPA runtime about what rego-version (v0/v1) to use to parse/compile contained Rego files.
The rego-version of individual Rego files can be overridden through the `file_rego_versions` manifest attribute.
Implements: #6578
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
In addition to those commands already supported:
* build
* check
* eval
* fmt
* test
support has been added to the following commands:
* `bench`
* `deps`
* `exec`
* `inspect`
* `parse`
* `run` (command `server` and `REPL`)
Fixes: #6520
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
This change adds a new flag to `opa run` to allow
users to specify a list of enabled TLS 1.0–1.2 cipher
suites. This allows users to control the cipher suites
the OPA server supports during a TLS handshake.
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
Regularly clean up of cache entries that have expired for a more efficient use of memory.
Introduce two new parameters to tune clean up frequency and threshold for forced FIFO eviction.
Fixes#5320
Signed-off-by: Rudrakh Panigrahi <rudrakh97@gmail.com>
This commit extends the telemetry report to include the
minimum compatible version of policies loaded into OPA.
This information can be helpful to get visibility into
era of Rego being adopted in the wild.
Fixes: #6361
Co-authored-by: Stephan Renatus <stephan@styra.com>
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
This change renames the `--future-compat` CLI flag on `opa run` to `v1-compatible`.
The latter is clear that it's a declaration that
gated features are compatible with a v1 version of OPA.
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
Currently OPA binds to the 0.0.0.0 interface by default, which allows
the OPA server to be exposed to services running outside of the same machine.
Though not inherently insecure in a trusted environment, it's good practice
to bind OPA to the localhost interface by default if OPA is not intended
to be exposed to remote services.
This change also adds a new feature flag to `opa run` to allow users to enable
future OPA compatible behavior.
Fixes: #6286
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
The partial query parameter has been marked as deprecated since
v0.23.0. It's also removed from the docs since that time.
Fixes: #2266
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
http_request_duration has fixed, hardcoded number of buckets with no possibility to tweak them
For cases when the most of the latencies are above 1ms, with only 4 available buckets there's no good insight on OPA's performance.
This implementation:
- adds the possibility for the buckets to be configurable in ```server.metrics.prom.http_request_duration_seconds.buckets``` key
- it's not a breaking change, if the buckets are not present in the configuration, the metric is configured with the existing values as a fallback
Signed-off-by: aarnautu <aarnautu@adobe.com>
The schema of the input document for the authorization
policy is known to OPA. This feature leverages that
to perform automatic type checking on the authorization policy.
The checks happen on policies provided to OPA on start-up and
also those provided via bundles. This check is enabled by default
and can be disabled using the `--skip-known-schema-check` flag
on `opa run`. This feature will help catch errors such as
typos, mismatch types etc. in these policies and provide precise
feedback to the policy author.
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
I have added a system for showing fatal and non-fatal deprecation warnings. It's configurable by command and environment.
If we merge this PR, running a rootless image with any OPA command other than `opa run` will result in a fatal error and exit code 1.
It's possible for users to continue to use the image by unsetting: OPA_DOCKER_IMAGE_TAG=rootless.
`opa run` will show the message, but it's not fatal for this command. This is intended to avoid production disruption.
Signed-off-by: Charlie Egan <charlie@styra.com>
Similar to the watch mode available in OPA when run as a server,
this change adds a watch mode in OPA test which
reloads the policy on file-system changes and re-runs the tests.
The watch mode in OPA test could be useful for example in TDD of
policies.
Fixes: #1719
Co-authored-by: Johan Fylling <johan.dev@fylling.se>
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
Currently if OPA listens on a Unix socket, the socket
file is created with 755 permissions. So if OPA is deployed
on k8s for example and the socket path is shared
via a volume between pods, due to the default permissions,
the socket will not be reachable for the caller. One way around
this is to match the user id for the OPA and caller containers
but that is not always possible.
This change adds a new flag to the OPA runtime that allows
to configure the permission of the socket file. In the
k8s scenario, if the file permission is updated to 777
for instance, the caller will be able to connect to OPA via
the socket.
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
Support OPA Client SDK programs loading bundles from an arbitraty filesystem, such as an in-memory filesystem, which unlocks additional uses that include compiling a bundle to an intermediate representation from a client program rather than the OPA command line.
Fixes#5833
bundle: Add filesystem support
Soften constraint in `Equal` method to support bundle comparison for rootless filesystems, eg treat "/file" and "file" as equal for both URLs and Paths
Add `WithPathFormat` for `DirectoryLoader` builders to centralise logic for how paths are returned during file traversal, ie in `NextFile`
Add support for specifiying the root directory for `dirLoaderFS`
compile: Add filesystem support
Add `WithFS` builder helper to pass into `initload.LoadPaths` to load bundles from a filesystem
internal/runtime/init: Add filesystem support
Pass newly supplied `fsys fs.FS` parameter in `LoadPaths` into file loader builder
loader: Add filesystem support
Add new `GetBundleDirectLoaderFS` which can load bundles from the supplied filesystem
runtime: Add filesystem support
Pass-through nil parameter as `fsys fs.FS` parameter into `initLoad.LoadPaths` (OPA servers/repls are not in scope for loading from filesystem)
util/test: Add in-memory filesystem support
Add new `WithTestFS` helper to allow tests that currently use `WithTempFS` to choose between a disk-based or memory-based filesystem - now used throughout `compile_test`
Signed-off-by: Kieran Othen <kieran.othen@mac.com>
Since all published OPA images now run with a non-root uid/gid,
there is no need to publish the rootless image tag. Currently
if the rootless variant is run, a log message at the Warn
level is printed.
This change increaes the log level for that message to Error
in order to make it more explict to stop using this variant in the
future.
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
run: added addrSetByUser to check if addr flag has been set by the user or not returns a bool to rt.ParamsAddrSetByUser in initRuntime
run_test: updated tests as we pass a bool now to initRuntime, added a test to check that changing the addr flag returns the expected output
runtime: added AddrSetByUser boolean and updated the server initialization info message, if AddrSetByUser is false, it will append an info message to the server initialization
runtime_test: added a test that will check the message is correctly set depending on if AddrSetByUser is true or false
docs/content/security: added paragraph addressing the potential issues of binding opa to the 0.0.0.0 interface
Fixes#5090
Signed-off-by: Peter Macdonald <macdonald.peter90@gmail.com>
The watch feature was removed ~2.5 years ago in
faab37c37e so it should be fine to
remove the corresponding log message.
Signed-off-by: Torin Sandall <torin@styra.com>
It is available for the following REST API endpoints:
- GET & POST HTTP methods on /v0/data & /v1/data endpoints
- POST HTTP method on /v1/compile endpoint
HTTP clients can optionally:
- send 'Accept-Encoding: gzip' header and expect a gzip compressed body and a Content-Encoding: gzip response header. The server will send the content encoded as gzip only after a threshold defined by server.encoding.gzip.min_length (default value is 1024). If the size is below the threshold, the body is not compressed
- send 'Content-Encoding: gzip' header and a gzip compressed body and expect the server to correctly interpret the request
Fixes#5310
Signed-off-by: aarnautu <aarnautu@adobe.com>
Currently all OPA image variants except "rootless" use
uid/gid 0 (i.e. root). Per container security best
practices it is better to run as non-root. So now OPA
defaults to non-root uid/gid in images. If root user
if needed, it can be explicitly set.
The "rootless" image variant is no longer needed and
will be not published in future releases.
Also currently the debug variant is published
for `linux/amd64` platform. For `linux/arm64` only static
images are generated. The debug variant can be useful
for debugging purposes and hence this change adds that
to the static image which can then be used on amd64 and arm64 arch.
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
Before, the capabilities were plumbled through in most places:
1. checking which builtins exist
2. passed along to the optimizer
3. passed along to the planner
But they hadn't been passed along to the file loader. As such, it could not
pass the caps along to the parser either. This is now done, but adding a new
method to the FileLoader interface.
Fixes#5323.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This is the OPA side of #4290. It will allow the envoy plugin to wire
the TraceProvider into the gRPC handlers.
Signed-off-by: vinhph0906 <vinhph0906@gmail.com>
And enable the `tenv` linter for the future.
Also, bump version of golangci-lint and fix some new
warnings that came from that.
Signed-off-by: Anders Eknert <anders@eknert.com>
Got a few warnings from my IDE about redundant type conversions,
so I decided to look into it. Added the unconvert linter to our
checks, and fixed the violations. Added two ignore comments as I
wasn't sure about whether they'd change the semantics of the code.
Signed-off-by: Anders Eknert <anders@eknert.com>
This commit adds support to the `compile` package for extracting
entrypoints from Rego `entrypoint` metadata annotations. The new
entrypoint annotations can be applied to any valid entrypoint target,
e.g. rules and packages.
An example policy using the new entrypoint annotations:
package test
# METADATA
# entrypoint: true
allow {
input.x
}
The `build` and `eval` CLI commands have been updated so that they do
not require an entrypoint to be explicitly provided with `-e`, so long
as at least one valid rule/package is marked with an entrypoint
annotation.
The new feature is additive; entrypoints can still be explicitly
provided to these commands by `-e`, and those will be used alongside the
entrypoints discovered from the metadata annotations.
Fixes: #3459
Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
Today it is not possible to correlate the decision log with
other types of logs (server, print, etc.) when the server log
level is >= INFO. The log correlation could be helpful in
troubleshooting.
A solution is to add a common attribute in all logs to make
the log correlation possible, so adding the req_id attribute
on decision logs, when server log level is >= INFO, will make it
possible.
Fixes: #5006
* Add documentation about decision log req_id attribute
The documentation purpose is to explain the relation with
others logs, how it could be used, and when it is included
on decision logs.
Signed-off-by: Humberto Corrêa da Silva <humbertoc_silva@hotmail.com>
This commit integrates the non-deterministic builtins caching system
into decision logging, both in the server and sdk packages. Some
reworking of the NDBCache's serialization format were required to
accommodate this. The feature is disabled by default, and must be
opted into by user configuration.
The feature can be enabled via a top-level config key:
nd_builtin_cache=true
The NDBCache is exposed to the masking system under the
`/nd_builtin_cache` path, which allows masking or dropping sensitive
values from decision logs selectively.
Note: If a decision log event exceeds the `upload_size_limit_bytes`
value for the OPA instance, OPA will reattempt uploading it, after
dropping the NDBCache from the event. This behavior will trigger a log
error, and will increment the `decision_logs_nd_builtin_cache_dropped`
metrics counter.
Fixes: #1514
Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
Johan Fylling