Go 1.23 is no longer supported as per Go release policy.
Changes:
- Use Go v1.24.6 as the project SDK requirement
- Apply lint fixes for Go 1.24
- Fix "non-constant format string in call" issues as seen in CI.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
All packages, except for `cmd` and `internal`, have been moved into a new `v1` root package.
Old packages are kept for backwards-compatibility reasons. All contained code is replaced with simple type aliases and proxy functions to `v1` implementations.
Old packages default to the Rego v0 syntax, new `v1` packages default to the Rego v1 syntax.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
To make OPA behave as v0.x post v1.0 release.
If used simultaneously with `--v1-compatible` flag, the `--v0-compatible` flag takes precedence.
Also, future-proofing `cmd` package tests for 1.0.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
This commit adds a new inter-query value cache that built-in
functions can use to cache information across queries.
For example, the `regex` and `glob` builtins can use this
to cache compiled regex and glob match patterns respectively.
The number of entries in the cache can be configured via the OPA
config. By default there is no limit.
Fixes: #6908
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
Previously if Discovery was enabled, other features like bundle downloading and status reporting could not be configured manually.
The reason for this was to prevent OPAs being deployed that could not be controlled through discovery. It's possible that
the system serving the discovered config is unaware of all options locally available in OPA. Hence, we relax the configuration
check when discovery is enabled so that the bootstrap configuration can contain plugin configurations. In case of conflicts,
the bootstrap configuration for plugins wins. These local configuration overrides from the bootstrap configuration are included
in the Status API messages so that management systems can get visibility into the local overrides.
**In general, the bootstrap configuration overrides the discovered configuration.** Previously this was not the case for all
configuration fields. For example, if the discovered configuration changes the `labels` section, only labels that are
additional compared to the bootstrap configuration are used, all other changes are ignored. This implies labels in the
bootstrap configuration override those in the discovered configuration. But for fields such as `default_decision`, `default_authorization_decision`,
`nd_builtin_cache`, the discovered configuration would override the bootstrap configuration. Now the behavior is more consistent
for the entire configuration and helps to avoid accidental configuration errors.
Fixes: #5722
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
Fixing issue where active parser options aren't propagated to module reload during bundle activation.
Signed-off-by: Francisco Rodrigues <ednofco@gmail.com>
This commit adds the possibility to configure the plugin manager with
custom options.
It will allow SDK users to override the options already provided by the
SDK and to futher customize it with configurations that were not
previously available. This is an advanced feature as it requires
some knowledge about the inner workings of OPA.
One use case for this is to provide a prometheus registerer and have the
status plugin metrics available for the client to use it in a
/metrics endpoint, for example.
resolves#6662
Signed-off-by: Francisco Rodrigues <ednofco@gmail.com>
In addition to those commands already supported:
* build
* check
* eval
* fmt
* test
support has been added to the following commands:
* `bench`
* `deps`
* `exec`
* `inspect`
* `parse`
* `run` (command `server` and `REPL`)
Fixes: #6520
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Regularly clean up of cache entries that have expired for a more efficient use of memory.
Introduce two new parameters to tune clean up frequency and threshold for forced FIFO eviction.
Fixes#5320
Signed-off-by: Rudrakh Panigrahi <rudrakh97@gmail.com>
Add two environment variable tests which illustrate how an environment variable can be used in OPA to verify a JWT.
Signed-off-by: Robert Hafner <robert.hafner@sas.com>
This allows clients to specify the decision ID for the decisions
returned by the `Decision` and `Partial` functions provided by the SDK
package. If not provided, a uniquely generated identifier will be
generated for the decision, just as before.
This option can be useful for clients that have already generated a
decision ID prior to calling OPA. It would then be convenient to pass
that decision ID through to OPA so decision logs use that same ID.
Signed-off-by: Brian Chhun <brian.chhun@chime.com>
The sdk package allows you to pass arbitrary Go objects as the input
document. This is fine for evaluation (as long as the object can be
parsed to an AST), but is problematic for decision log masking, as the
masking logic expects the input on the event to be either a
map[string]interface{} or a []interface{}, and for inner types of the
object to also be similarly generic. Currently, the decision log masking
silently fails if the input object is not of the correct type.
This patch addresses this by rebuilding the input Go type for the
decision log from the parsed AST of the original input object. This
generates a Go type that does not break the masking logic, provided the
input is of a type that can be masked. The conversion from the AST only
happens if the decision logs plugin is actually registered with the
manager to avoid wasting cycles if decision logs are not enabled.
A test is added to cover the masking case for the SDK.
Signed-off-by: Erik Paulson <epaulson10@gmail.com>
Currently we parse store modules irrespective of whether
there are modules on the Rego object. This will result in
the compilation of those modules which triggers the bundle
activation flow. Now as part of the module compilation
we interact with the compiler's modules and run compilation
on the input modules. If let's say there are concurrent health
check requests (ie. /v1/health), this could result in a race
during the compilation process while working with the compiler's
modules.
This change avoids this situation by skipping parsing of the store
modules when none are set on the Rego object. The assumption this
change makes is that while using the rego package the compiler and
store are kept in-sync.
Fixes: #5868
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
When the discovery plugin receives a a discovery bundle which omits the
discovery configuration, it deletes its own configuration from the
manager. In turn this means that `GET /v1/config` doesn't show the
configuration of the discovery plugin.
This change ensures that the plugin will never overwrite the discovery
configuration on the manager.
Signed-off-by: Benjamin Nørgaard <mail@blacksails.dev>
This adds a lightweight extensibility mechanism to OPA: hooks. Loosely
modelled on what franz-go supports (see refs below).
We're starting with a configuration hook. It allows us to inspect or
alter the configuration of OPA after...
1. the config is read and parsed: OnConfig
2. a discovery bundle is processed: OnConfigDiscovery
References:
- franz-go: https://pkg.go.dev/github.com/twmb/franz-go/pkg/kgo#Hook
To follow:
- more hooks where they are useful
- runtime support for hooks, wiring them into the proper other places
Signed-off-by: Stephan Renatus <stephan@styra.com>
The store to use in the SDK is currently hard-coded to be inmem.
This change allows providing new implementations.
Signed-off-by: Stephan Renatus <stephan@styra.com>
* [rego] Check store modules before skipping parsing
Fixes https://github.com/open-policy-agent/opa/issues/5511
This change will cause the operation to be timed if the store
modules have already all been compiled and there are no
rawModules. This might be undesirable.
Signed-off-by: Charlie Egan <charlie@styra.com>
This will allow users of the SDK to see the version information of the
bundles used in decisions as well as the OPA SDK version.
Signed-off-by: Charlie Egan <charlieegan3@users.noreply.github.com>
This allows the struct builtin errors functionality to be used in the SDK by passing the value in DecisionOptions & PartialOptions.
Related to #5176
Signed-off-by: Charlie Egan <charlieegan3@users.noreply.github.com>
This commit integrates the non-deterministic builtins caching system
into decision logging, both in the server and sdk packages. Some
reworking of the NDBCache's serialization format were required to
accommodate this. The feature is disabled by default, and must be
opted into by user configuration.
The feature can be enabled via a top-level config key:
nd_builtin_cache=true
The NDBCache is exposed to the masking system under the
`/nd_builtin_cache` path, which allows masking or dropping sensitive
values from decision logs selectively.
Note: If a decision log event exceeds the `upload_size_limit_bytes`
value for the OPA instance, OPA will reattempt uploading it, after
dropping the NDBCache from the event. This behavior will trigger a log
error, and will increment the `decision_logs_nd_builtin_cache_dropped`
metrics counter.
Fixes: #1514
Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
This commit adds the `prealloc` linter to the list of linters for OPA, and fixes up the miscellaneous locations in the code that the linter found where we could easily preallocate slices.
Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
When calling the sdk the user can specify an opa id so that repeat calls to the sdk don't have a unique opa id
Signed-off-by: PrahathessRaghavan <prahathess@gmail.com>
When running on GHA, we've found this test to often fail on macos-latest:
It would not functionally be wrong, but it also wasn't able to finish in
the alloted time. Now, the maximum delta has been increased a lot (10ms to
500ms). It's much, but it's still good enough to ensure that the context
passed to Stop() is the one that matters for shutdown.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
I noticed that when operating on opa.state, locking is usually done to avoid
a race, whereas here opa.state is used directly. By comparing the previous
changes, I found that #4240 changed the previous behaviour.
This change adjusts that: we ensure that we work on the state as read via
the mutex-protected `s := *opa.state`.
Signed-off-by: Iceber Gu <wei.cai-nat@daocloud.io>
Adding the ability to partially evaluate when using the the SDK as a go library.
This allows for utilizing the existing OPA configuration (e.g. bundle, decisions,
etc) when partially evaluating.
Signed-off-by: Kurt Roekle <kroekle@gmail.com>
* sdk/opa_test: increase max delta
6ms was arbitrary, and so is 10ms.
When GHA switched the macos-latest version, we've started seeing
test flakiness here.
30ms (20ms+10ms) are still waaaay below the 1s that the test_plugin
attempts need to shut down.
* test/e2e/certrefresh: double wait time for macos runner
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This commit updates the SDK to send print output to the logger if the
level is INFO or DEBUG and disables print statements entirely
otherwise. This way the SDK behaves teh same was as the server.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
The test server expects mock bundle files to be prefixed with
/bundles/ so return an error if they are not (otherwise the user has
to dig into the test server code to figure out why their test is not working.)
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
With this change, the manager will respect the shutdown period if
it was supplied, otherwise it will use the passed context. This way,
SDK users can rely on the context (because the SDK doesn't set
the graceful shutdown period), but other callers are unaffected.
The added test is in the SDK, because that's where the problem had
manifested (#3980): when calling Stop(ctx) through the SDK, the
plugins.Manager's Stop function had set a smaller timeout (0, due
to the structs default value that hadn't been set to anything else),
and that would effectively leave the plugins zero time to cleanup.
Fixes#3980.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This adds a Plugins field to the SDK Options struct that accepts a
map[string]plugins.Factory similarly to the map created by
runtime.RegisterPlugin this map is passed to the discovery.Factories
plugin similarly, again, to how it works within the Runtime module. This
allows an OPA SDK instance to initialize plugins.
Fixes#3826
Signed-off-by: Edward Paget <edward.paget@chime.com>
Fixed changes requested by @tsandall (squash before merge)
Signed-off-by: Edward Paget <edward.paget@chime.com>
This commit fixes the console loggers so that messages are emitted
regardless of the debug log level. The problem was that in 3fcc875 we
updated the plugins to use a console logger obtained from the plugin
manager as opposed to a global logger instantiated in the plugins
package--the console logger obtained from the plugin manager was
instantiated in the runtime package by calling
logging.NewStandardLogger. Unfortunately, logging.NewStandardLogger
does not create a new logger--it returns the global logrus
logger.
This commit fixes the issue by deprecating logging.NewStandardLogger
and introducing two new functions in the logging package:
* logging.Get() - this replaces the old logging.NewStandardLogger
function--this function should be called to obtain the debug logger
used throughout OPA.
* logging.New() - this actually returns a new logger that can be
configured independently from the debug logger used throughout
OPA.
The runtime and sdk packages have been updated to call logging.New()
to obtain console loggers and the rest of the codebase has been
updated to call logging.Get() in place of logging.NewStandardLogger().
Fixes#3654
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
golint is deprecated. The author of the code no longer supports the
codebase. golangci-lint is faster than golint, and is in use by other
opa repositories (e.g. Gatekeeper).
This commit changes tools.go to reference golangci (so it ends up in
vendor) and modifies check-lint to use golangci instead.
Breaking API Changes:
- plugins/rest/rest.go: Fix typo "AllowInsureTLS" -> "AllowInsecureTLS"
- storage/errors.go: Removed unused IndexingNotSupportedErr
Signed-off-by: Will Beason <willbeason@google.com>
This commit updates the test server so that callers can control when
the server becomes ready. This allows us to test the async option on
the SDK without worrying about race conditions. I.e., we can block
server readiness and assert that the SDK returns undefined until ready.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Ville Vesilehto