❗ We now parse rego metadata annotations by default.
Rule annotations now support a `labels` field. During policy eval,
labels from all successfully evaluated rules are collected and included
in each decision log entry as a top-level `rule_labels` array. Each
element preserves the label map from one evaluated rule. Exact
duplicates are omitted.
```rego
# METADATA
# labels:
# severity: low
# team: platform
allow if input.role == "admin"
```
The resulting decision log entry will contain:
```json
{"rule_labels": [{"severity": "low", "team": "platform"}]}
```
---------
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Rules can now be annotated with a metadata `id` field. When any
metadata `id` annotations are present in the rego (scope: rule), the IDs
of successfully evaluated rules are included in decision log events.
Additionally, the Data API supports a `?id` query parameter to
include evaluated rule IDs directly in the response payload.
```rego
# METADATA
# id: allow-admin
allow if input.role == "admin"
```
Modules containing `id` annotations will have metadata parsing enabled
automatically.
Fixes#2089
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
The plugin registration step was missing before, so the code, while in
the tree, was not active and the plugin couldn't be used.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
External rule sources let wrapping projects inject rules at evaluation
time instead of compile time. The compiler marks external packages in
the rule tree but doesn't index them. When topdown hits an external
node, it calls Lookup to get rules, compiles them on the fly with a
scoped compiler, grafts the result into the tree, and caches it for the
duration of the evaluation.
Sources can be isolated (default, no access to surrounding policy) or
non-isolated (can reference static rules and other external sources).
The ExternalRuleIndexCloser interface handles cleanup after evaluation.
Precompiled rules can skip compiler stages via SkippedStages to avoid
redundant work.
This includes:
* hooks: add BundlePreActivate hook This one is handy when registering
external sources.
* topdown: catch `ir == nil` rule index result
This wouldn't ordinarily happen: the compiler is checking refs before.
But in our use case, the SP rules may be configured to be able to reach
into the surrounding Rego (non-isolated mode). If that happens, the IR
lookup may indeed end up as `nil, nil`.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
The BufferedLogger introduced for logger plugins is created at startup
and passed to the `*plugins.Manager`. Plugins (bundle, discovery,
status, logs) cache `manager.Logger()` in a field at construction time.
After `Manager.Start()`, `ResolveBufferedLogger` flushes the buffer and
swaps the `Manager'`s logger to a `StandardLogger` — but the plugins
still hold the old `BufferedLogger`. Since bundle loading is async, the
"Bundle loaded and activated successfully" message (and similar) gets
written to the already-flushed buffer where nobody reads it.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
- The tokenTLSConfig is cloned from DefaultTLSConfig(c) (which
includes RootCAs) and then InsecureSkipVerify is set explicitly from
c.AllowInsecureTLS, since DefaultTLSConfig only sets it when the service
URL is https, but the token URL is always https regardless.
- New test configures a service with tls.ca_cert pointing to the token
server's CA cert, without allow_insecure_tls. This directly reproduces
the bug report — before the fix, this would fail with x509: certificate
signed by unknown authority. Also removes some duplication we had
before.
Fixes#8473.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This PR brings support for pluggable logging implementations via the logger plugin interface, which is based on Go's standard log/slog.Handler interface. This allows any slog.Handler implementation to be used as a logger plugin. Loggers can be referenced via the server.logger_plugin configuration option; and can also be used for decision logs. OPA includes a built-in file logger plugin (file_logger) that writes structured JSON logs with rotation support using lumberjack. Users can also implement and register custom logger plugins when building OPA.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Added support for web identity credentials in the AWS signing plugin.
This allows users to configure the plugin to use a web identity token file
for authentication when assuming a role,
which is particularly useful in environments like Kubernetes
where service accounts can be used to provide AWS credentials.
Signed-off-by: Tiago Viegas <tiago.viegas@siemens.com>
Co-authored-by: Johan Fylling <johan.dev@fylling.se>
* plugins/rest: cache *http.Client and auth plugin
This will require further changes to cert TLS and token auth methods to
stay compatible with the previous behaviour.
* plugins/rest: configurable re-read interval for TLS cert+key
Defaulting to re-reading all the time, more or less like we did before.
(I write "more or less" because we now do it in `GetClientCertificate()`.)
* plugins/rest: document change (code comments, CHANGELOG)
* plugins/rest: set minimum TLS version where `&tls.Config{}` is used
* plugins/rest: ensure min TLS version and ciphersuites are used
...as configured with the server.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
And enable more staticcheck linters. I saw staticcheck failures
mentioned in another PR, so thought I'd check it out.
- `WriteString(fmt.Sprintf)` -> `fmt.Fprintf`
- Rewrite calls to deprecated `*Rule.Path()`
- Don't use `==` to compare `time.Time`
- Use inline ignores over config exclusions of paths
- Remove 'varcheck' ignores as no longer used
- Remove v0 topdown/graphql.go (!)
Signed-off-by: Anders Eknert <anders.eknert@apple.com>
1. We no longer resolve the Auth plugin twice (once for *http.Client,
once for Prepare()).
2. Avoid an intermediate map for request headers because we can set it
right away (semantically equivalent via Set, not Add)
3. Simplify masked header creation
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This introduces a new trigger mode for the decision log plugin:
decision_logs.reporting.trigger=immediate
The immediate trigger mode will upload events as soon as enough events are received to hit the configured upload limit. If not enough events are received within the configured min-max delay, the events received so far are flushed and uploaded.
Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
Add a separate field to keep track of the parsed min and max interval keeping the original fields to represent the user provided values. This prevents multiple calls to config.ValidateAndInjectDefaults from overflowing the values but still makes sure that the value has been converted at least once.
Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
* runtime: Correct naming of version checking code
Rename telemetry functionality to version checking to accurately reflect
current behavior following
https://github.com/open-policy-agent/opa/pull/7756.
The system only checks GitHub releases for version updates without sending
any data about the OPA instance and so the privacy docs have been updated too.
Signed-off-by: Charlie Egan <charlie_egan@apple.com>
* Make WithTelemetryGatherers a no-op
Deprecate WithTelemetryGatherers since telemetry gathering has been removed.
The function now returns a no-op to maintain API compatibility without
breaking existing code that might uses it.
Signed-off-by: Charlie Egan <charlie_egan@apple.com>
---------
Signed-off-by: Charlie Egan <charlie_egan@apple.com>
The test was starting the plugin's event loop which spawns a goroutine
that consumes from bulkBundleCh. This raced with the test's channel
length assertion, causing intermittent failures on CI.
Since the test verifies PushFIFO buffer semantics rather than the
plugin lifecycle, remove the unnecessary Start/Stop calls to eliminate
the race.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
decisions are being dropped because after an upload the limit is being set to the upload size instead of the buffer.
Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
Adds initialisation for registered plugins when using discovery
bundles without persistence configured.
This allows discovery bundle downloaders to use custom credential plugins
Signed-off-by: James Turner <jt28828@gmail.com>
- Bump golangci-lint -> 2.6.2
- Fix all `deprecatedComment` "notices should be in a dedicated paragraph, separated from the rest" reports
- Enable `appendCombine` and fix all "appendCombine: can combine chain of X appends into one" notices
- Enable `preferFprint` and fix the few reported issues
- Fix various issues reported only once or twice, like `zeroByteRepeat`
Signed-off-by: Anders Eknert <anders.eknert@apple.com>
Have done this some time in the past, but there was a few
new issues this would highlight now that we're on Go 1.24.
Mostly:
- Use `b.Loop()` in benchmarks
- Use `strings.SplitSeq` where possible
- Remove `omitempty` tag for types that can't be empty
Signed-off-by: Anders Eknert <anders@eknert.com>
In the encoder, if the event size equals the limit it was added to the buffer.
Instead return it to avoid unnecessary downsize step that could lead to an infinite loop.
Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
Updates the Bundle Plugins oneShot callback function signature used by Downloader, OCIDownloader, and fileLoader to return an error. This allows any issues in the callback function such as Rego parsing issues to be returned.
Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
refactors the original buffer implementation (now labelled as the size buffer) to implement the same interface as the event buffer type instead of being baked into the Plugin type. This helps create a clearer distinction between the two buffer types.
Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
Go 1.23 is no longer supported as per Go release policy.
Changes:
- Use Go v1.24.6 as the project SDK requirement
- Apply lint fixes for Go 1.24
- Fix "non-constant format string in call" issues as seen in CI.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
* v1/plugins: Address race in config access
I ran into this race condition on another PR:
https://github.com/open-policy-agent/opa/actions/runs/16655603110/job/47139789057
I have tried to make all manager.Config access thread-safe by adding new
getters for used values. GetConfig is regrettably based on a JSON
roundtrip deep copy of the config. This us used in tests (fine) but also
in the discovery plugin:
https://github.com/open-policy-agent/opa/blob/2d014a89bbbc307d7204817220146ffae992e838/v1/plugins/discovery/discovery.go#L122
getPluginSet is very tightly coupled to the manager.Config and because
of it's dependencies on status and the other plugins packages, it's hard
to break out.
So, for now, I think this is an improvement and worth getting a second
opinion on before more refactoring.
Signed-off-by: Charlie Egan <charlie@styra.com>
* v1/config: Use add Clone to config
This makes the use of the manager's config more thread-safe and
consistent without more API changes.
Signed-off-by: Charlie Egan <charlie@styra.com>
* topdown: Add clone() funcs for config structs
NamedValueCacheConfig.Clone, InterQueryBuiltinValueCacheConfig.Clone and
InterQueryBuiltinCacheConfig.Clone have been added.
All Clone methods return a deep copy of the struct. This is tested for
missed new fields using PopulateAllFields, a generic function that
stuffs structs with values for all fields.
Signed-off-by: Charlie Egan <charlie@styra.com>
* plugins: Clone new config
Signed-off-by: Charlie Egan <charlie@styra.com>
---------
Signed-off-by: Charlie Egan <charlie@styra.com>
* plugin/bundle: Correct bundle delay behavior
I ran into an issue when testing an earlier change:
https://github.com/open-policy-agent/opa/actions/runs/16646981900/job/47110035165
I found that this test generated around 100,000 lines of errors showing
the bundle downloader running.
This can be be tested using:
```
go test -v ./v1/plugins/bundle -count=1 2>&1 | grep -c "request failed"
```
This commit closes managers and plugins correctly.
Signed-off-by: Charlie Egan <charlie@styra.com>
* download: Update stop to be idempotent
I had some race detector issues with TestStartStopWithLongPollNotSupported
https://github.com/open-policy-agent/opa/actions/runs/16722869930/job/47334690407?pr=7812
I think this is a deadlock around multiple calls to Stop dead locking
updating the stopped var.
Signed-off-by: Charlie Egan <charlie@styra.com>
---------
Signed-off-by: Charlie Egan <charlie@styra.com>
This commit adds an experimental "intermediate results" field to
decision logs, and provides some basic plumbing in the server package
for attaching the intermediate results of an eval to the request
context.
Co-authored-by: Teemu Koponen <koponen@styra.com>
Signed-off-by: Philip Conrad <philip@chariot-chaser.net>
This commit provides an extension mechanism for the server authorizer,
allowing plugins and other server extensions to inform the authorizer
about the methods and paths where it should expect and parse request
bodies.
Signed-off-by: Philip Conrad <philip@chariot-chaser.net>
This commit adds a new field to Decision Log entries, allowing batches
of decisions to be correlated together later.
Signed-off-by: Philip Conrad <philip@chariot-chaser.net>
Before we had introduced `http.ServeMux` as "the router", we had been
using github.com/gorilla/mux. Using the latter, it was possible to
inject middlewares using the mux's `.Use()` method. This mechanism
allowed global middlewares to be injected from `runtime.Params`, for
example.
With `http.ServeMux`, that's no longer possible. However, it was never
an intentionally supported feature in the first place.
So this commit introduces HTTP handler middlewares as extension points.
It's modelled after `(*plugins.Manager).ExtraRoute()`.
Signed-off-by: Stephan Renatus <stephan@styra.com>
This allows simple setups -- those feeding the OPA discovery plugin with
a static JSON file -- to still do env variable replacements.
This should be possible already, by using a policy to construct the
disco config, but it becomes easier now.
Co-authored-by: Teemu Koponen <koponen@styra.com>
Signed-off-by: Stephan Renatus <stephan@styra.com>
This commit adds support for changing out how bundle storage and
activation work. To allow swapping out bundle activation, two new
`bundle` package functions are provided:
- `RegisterActivator`: Registers a bundle.Activator with a string ID.
- `RegisterDefaultBundleActivator`: Sets the default bundle.Activator to
use by ID.
Behind the scenes, a few new `bundle` package variables are used to
track what bundle activators are available, and which is the preferred
default.
This system allows registering many activators, and allows choosing the
bundle activator to use at activation time. The activator to use is
decided in the following order:
- `(bundle.ActivateOpts).Plugin` is used when non-nil.
- `bundle.bundleExtActivator` is used when an ID was set with
`RegisterDefaultBundleActivator`.
- The default/original bundle activator is used if no other selection
was made.
To support swapping out bundle storage (useful when testing new bundle
designs), a new `bundle` package function is provided:
- `RegisterStoreFunc`: Sets the function to use for creating bundle
storage.
These two features together allow swapping out most of the bundle
activation flow, without requiring deep modification of the `bundle`
package. Lazy bundle loading mode is also enabled across many CLI
commands and other bundle loading points now when a non-default bundle
activator is set.
Signed-off-by: Philip Conrad <philip@chariot-chaser.net>
Co-authored-by: Ashutosh Narkar <anarkar4387@gmail.com>
Otherwise, setting something from the runtime parameter
ExtraDiscoveryOpts would be impossible: on runtime startup, the runtime
is injecting its own registered plugins via that method.
With this change, for example factories passed via discovery.Factories()
in ExtraDiscoveryOpts will be able to add to (or replace) the previously
registered plugins.
Signed-off-by: Stephan Renatus <stephan@styra.com>
This way, the extra handler functions are still covered by prometheus
metrics and opentelemetry spans.
The previous method of directly registering routes with the router
bypassed the server's handler wrapping.
Signed-off-by: Stephan Renatus <stephan@styra.com>
Stephan Renatus