Part of #2745
Like most of his ideas, @anderseknert's suggestion to use Rego to
replace the `validateAndInjectDefaults` functions throughout the
codebase is another winner.
This PR starts the migration by replacing the top-level
`validateAndInjectDefaults` in `v1/config/config.go` with an embedded
policy, `validate.rego`. The policy injects the top-level defaults
(`default_decision`, `default_authorization_decision`, `labels`) and
reports unrecognized configuration options, so a typo such as
`decision_log` instead of `decision_logs` is logged as a warning at
startup rather than silently ignored.
It's evaluated in `ParseConfig` using the low-level `ast`/`topdown`
packages rather than the top-level `rego` package. This keeps `config`
off the heavy `rego → bundle → …` dependency web (which would otherwise
create import cycles as more packages' tests reach `config`), and we
don't need any of the `rego` package's conveniences here — it's one
module compiled once and a single query. The Rego unit tests run in CI
via `build/run-rego-tests.sh` (and locally with `make rego-test`).
This sets the foundation for the other plugin
`validateAndInjectDefaults` functions to migrate to Rego as well; where
the logic isn't too complicated it should be a fairly easy replacement.
At the moment all known keys live in `validate.rego` under `_specs` to
support the "warn on unrecognized options" check, but the
plugin-specific entries can move closer to each plugin as it migrates.
It would also be nice for `_specs` to be auto-generated somehow in the
future.
Supporting extension of config validation with custom policies is
something I'd like to follow up with, so keeping #2745 open for now.
I also think these policies could be reusable with
[java-opa-sdk](https://github.com/open-policy-agent/java-opa-sdk) 👀
Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
The memory pressure in low resource containers this PR aimed to fix was
actually caused by #8817, which was fixed in #8829. The
automaxprocs/automemlimit dependencies are no longer needed as was
intended in #8696.
This reverts commit 88c01e659c and updates
docs to match the current behaviour.
---------
Signed-off-by: Charlie Egan <charlie_egan@apple.com>
This is handled natively by Go since 1.25, so this dependency should no
longer be needed. See references below for more information. Only
notable difference seems to be that Go sets a minimum value of 2 while
the automaxprocs lib has a minimum value of 1. Go seems to account for
much more though, so I don't think that difference alone warrants the
inclusion of this dependency. Users who really want GOMAXPROCS=1 can
always set that themselves.
References:
- https://github.com/golang/go/issues/73193
- https://github.com/uber-go/automaxprocs/issues/98
Signed-off-by: Anders Eknert <anders.eknert@apple.com>
❗ We now parse rego metadata annotations by default.
Rule annotations now support a `labels` field. During policy eval,
labels from all successfully evaluated rules are collected and included
in each decision log entry as a top-level `rule_labels` array. Each
element preserves the label map from one evaluated rule. Exact
duplicates are omitted.
```rego
# METADATA
# labels:
# severity: low
# team: platform
allow if input.role == "admin"
```
The resulting decision log entry will contain:
```json
{"rule_labels": [{"severity": "low", "team": "platform"}]}
```
---------
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
The plugin registration step was missing before, so the code, while in
the tree, was not active and the plugin couldn't be used.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
* distributedtracing: export Prometheus metrics via OTLP
Add support for pushing OPA's existing Prometheus metrics to an
OpenTelemetry collector via OTLP, eliminating the need for a dedicated
scraper sidecar. Uses the OTel Prometheus bridge to read from OPA's
prometheus.Registry and export through an OTLP metric exporter (gRPC
or HTTP), reusing the same address and TLS configuration as traces.
New config fields: distributed_tracing.metrics (bool, default false)
and distributed_tracing.metrics_export_interval_ms (int, default 60000).
Fixes#7591
Signed-off-by: Michael Munch <mm.munk@gmail.com>
* metricsexport: decouple metrics export into top-level config section
Extract metrics export from distributed_tracing into its own
metrics_export config section with independent type (otlp/grpc,
otlp/http), address, and TLS settings. This allows exporting
Prometheus metrics via OTLP without enabling tracing, and to a
different endpoint than traces.
- Extract shared TLS helpers into internal/tlsutil
- Add MetricsExport field to top-level Config
- Create internal/metricsexport package with Init, config parsing
- Remove metrics fields from distributedtracing
- Update runtime to call metricsexport.Init separately
- Move e2e tests to v1/test/e2e/metricsexport
- Add Metrics Export section to configuration docs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Michael Munch <mm.munk@gmail.com>
* ci: retrigger checks
Signed-off-by: Michael Munch <mm.munk@gmail.com>
* go.mod: upgrade dependencies downgraded during rebase
Modules like containerd, go-sqlbuilder, OpenTelemetry, and golang.org/x/*
were at older versions than main after a rebase. Upgrade them to match or
exceed main.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Michael Munch <mm.munk@gmail.com>
* Update internal/distributedtracing/distributedtracing_test.go
Signed-off-by: Michael Munch <mm.munk@gmail.com>
---------
Signed-off-by: Michael Munch <mm.munk@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This PR brings support for pluggable logging implementations via the logger plugin interface, which is based on Go's standard log/slog.Handler interface. This allows any slog.Handler implementation to be used as a logger plugin. Loggers can be referenced via the server.logger_plugin configuration option; and can also be used for decision logs. OPA includes a built-in file logger plugin (file_logger) that writes structured JSON logs with rotation support using lumberjack. Users can also implement and register custom logger plugins when building OPA.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
* plugins/rest: cache *http.Client and auth plugin
This will require further changes to cert TLS and token auth methods to
stay compatible with the previous behaviour.
* plugins/rest: configurable re-read interval for TLS cert+key
Defaulting to re-reading all the time, more or less like we did before.
(I write "more or less" because we now do it in `GetClientCertificate()`.)
* plugins/rest: document change (code comments, CHANGELOG)
* plugins/rest: set minimum TLS version where `&tls.Config{}` is used
* plugins/rest: ensure min TLS version and ciphersuites are used
...as configured with the server.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
And enable more staticcheck linters. I saw staticcheck failures
mentioned in another PR, so thought I'd check it out.
- `WriteString(fmt.Sprintf)` -> `fmt.Fprintf`
- Rewrite calls to deprecated `*Rule.Path()`
- Don't use `==` to compare `time.Time`
- Use inline ignores over config exclusions of paths
- Remove 'varcheck' ignores as no longer used
- Remove v0 topdown/graphql.go (!)
Signed-off-by: Anders Eknert <anders.eknert@apple.com>
Implements RegisterStorageBackend() to allow Go module users to inject custom
storage implementations during early-stage package init (hardcoded at build time).
Previously this was only possible through params.StoreBuilder which required
direct SDK usage.
Key changes:
- Add RegisterStorageBackend() and StorageBackendBuilder type in v1/runtime
- Modify storage initialization to check registered backends
Fixes#8277
Signed-off-by: alex60217101990 <alex6021710@gmail.com>
* runtime: Correct naming of version checking code
Rename telemetry functionality to version checking to accurately reflect
current behavior following
https://github.com/open-policy-agent/opa/pull/7756.
The system only checks GitHub releases for version updates without sending
any data about the OPA instance and so the privacy docs have been updated too.
Signed-off-by: Charlie Egan <charlie_egan@apple.com>
* Make WithTelemetryGatherers a no-op
Deprecate WithTelemetryGatherers since telemetry gathering has been removed.
The function now returns a no-op to maintain API compatibility without
breaking existing code that might uses it.
Signed-off-by: Charlie Egan <charlie_egan@apple.com>
---------
Signed-off-by: Charlie Egan <charlie_egan@apple.com>
- Bump golangci-lint -> 2.6.2
- Fix all `deprecatedComment` "notices should be in a dedicated paragraph, separated from the rest" reports
- Enable `appendCombine` and fix all "appendCombine: can combine chain of X appends into one" notices
- Enable `preferFprint` and fix the few reported issues
- Fix various issues reported only once or twice, like `zeroByteRepeat`
Signed-off-by: Anders Eknert <anders.eknert@apple.com>
Have done this some time in the past, but there was a few
new issues this would highlight now that we're on Go 1.24.
Mostly:
- Use `b.Loop()` in benchmarks
- Use `strings.SplitSeq` where possible
- Remove `omitempty` tag for types that can't be empty
Signed-off-by: Anders Eknert <anders@eknert.com>
Go 1.23 is no longer supported as per Go release policy.
Changes:
- Use Go v1.24.6 as the project SDK requirement
- Apply lint fixes for Go 1.24
- Fix "non-constant format string in call" issues as seen in CI.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
* v1/plugins: Address race in config access
I ran into this race condition on another PR:
https://github.com/open-policy-agent/opa/actions/runs/16655603110/job/47139789057
I have tried to make all manager.Config access thread-safe by adding new
getters for used values. GetConfig is regrettably based on a JSON
roundtrip deep copy of the config. This us used in tests (fine) but also
in the discovery plugin:
https://github.com/open-policy-agent/opa/blob/2d014a89bbbc307d7204817220146ffae992e838/v1/plugins/discovery/discovery.go#L122
getPluginSet is very tightly coupled to the manager.Config and because
of it's dependencies on status and the other plugins packages, it's hard
to break out.
So, for now, I think this is an improvement and worth getting a second
opinion on before more refactoring.
Signed-off-by: Charlie Egan <charlie@styra.com>
* v1/config: Use add Clone to config
This makes the use of the manager's config more thread-safe and
consistent without more API changes.
Signed-off-by: Charlie Egan <charlie@styra.com>
* topdown: Add clone() funcs for config structs
NamedValueCacheConfig.Clone, InterQueryBuiltinValueCacheConfig.Clone and
InterQueryBuiltinCacheConfig.Clone have been added.
All Clone methods return a deep copy of the struct. This is tested for
missed new fields using PopulateAllFields, a generic function that
stuffs structs with values for all fields.
Signed-off-by: Charlie Egan <charlie@styra.com>
* plugins: Clone new config
Signed-off-by: Charlie Egan <charlie@styra.com>
---------
Signed-off-by: Charlie Egan <charlie@styra.com>
I saw a panic here: https://github.com/open-policy-agent/opa/actions/runs/16652495113/job/47128828522
and think it's related to the server init check returning before the
addrs are set.
I update all cases where a similar check on logs is done.
I am unsure how the log could come before the server is initialized, but
sometimes funny things happen in race detector ordering and this looks
more correct to me.
Signed-off-by: Charlie Egan <charlie@styra.com>
This change allows users that build their own executable or "spin" of
OPA to give it a name, and have it reference itself properly in help
texts.
It's a vanity thing, but I think some people would appreciate it, hat
tip to the international association of pedants.
Signed-off-by: Stephan Renatus <stephan@styra.com>
Co-authored-by: kevinstyra <83973046+kevinstyra@users.noreply.github.com>
This commit provides an extension mechanism for the server authorizer,
allowing plugins and other server extensions to inform the authorizer
about the methods and paths where it should expect and parse request
bodies.
Signed-off-by: Philip Conrad <philip@chariot-chaser.net>
Before we had introduced `http.ServeMux` as "the router", we had been
using github.com/gorilla/mux. Using the latter, it was possible to
inject middlewares using the mux's `.Use()` method. This mechanism
allowed global middlewares to be injected from `runtime.Params`, for
example.
With `http.ServeMux`, that's no longer possible. However, it was never
an intentionally supported feature in the first place.
So this commit introduces HTTP handler middlewares as extension points.
It's modelled after `(*plugins.Manager).ExtraRoute()`.
Signed-off-by: Stephan Renatus <stephan@styra.com>
`os.Exit` immediately exits the program and doesn't run defer functions.
This can be problematic as any command.OnFinalize routines and any logic
after the command.Execute won't be run.
Also suppress all RunE cobra error and usage messages. These would be
printed twice otherwise.
Signed-off-by: Stephan Renatus <stephan@styra.com>
Co-authored-by: Kevin St. Pierre <kevin@styra.com>
This commit adds support for changing out how bundle storage and
activation work. To allow swapping out bundle activation, two new
`bundle` package functions are provided:
- `RegisterActivator`: Registers a bundle.Activator with a string ID.
- `RegisterDefaultBundleActivator`: Sets the default bundle.Activator to
use by ID.
Behind the scenes, a few new `bundle` package variables are used to
track what bundle activators are available, and which is the preferred
default.
This system allows registering many activators, and allows choosing the
bundle activator to use at activation time. The activator to use is
decided in the following order:
- `(bundle.ActivateOpts).Plugin` is used when non-nil.
- `bundle.bundleExtActivator` is used when an ID was set with
`RegisterDefaultBundleActivator`.
- The default/original bundle activator is used if no other selection
was made.
To support swapping out bundle storage (useful when testing new bundle
designs), a new `bundle` package function is provided:
- `RegisterStoreFunc`: Sets the function to use for creating bundle
storage.
These two features together allow swapping out most of the bundle
activation flow, without requiring deep modification of the `bundle`
package. Lazy bundle loading mode is also enabled across many CLI
commands and other bundle loading points now when a non-default bundle
activator is set.
Signed-off-by: Philip Conrad <philip@chariot-chaser.net>
Co-authored-by: Ashutosh Narkar <anarkar4387@gmail.com>
* storage: allow overriding NonEmpty
Custom store implementations can now bring their own NonEmpty() methods,
which may be more efficient than what the generic method does.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
* runtime: allow passing in custom store builder
Signed-off-by: Stephan Renatus <stephan@styra.com>
---------
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Signed-off-by: Stephan Renatus <stephan@styra.com>
Otherwise, setting something from the runtime parameter
ExtraDiscoveryOpts would be impossible: on runtime startup, the runtime
is injecting its own registered plugins via that method.
With this change, for example factories passed via discovery.Factories()
in ExtraDiscoveryOpts will be able to add to (or replace) the previously
registered plugins.
Signed-off-by: Stephan Renatus <stephan@styra.com>
When an plugin http handler or some other mechanism wants to do rego
evaluations, too, it's beneficial to share the caches with the server.
This change introduces two new hook types to allow retrieving those
caches during server startup.
Signed-off-by: Stephan Renatus <stephan@styra.com>
This commit comprehensively plumbs in the bundle lazy loading mode
option in the compile, runtime, rego, and bundle packages. It also
includes the bare minimum plumbing to allow the path watcher utilities
to also toggle the option on.
In nearly all places where a default is expected, the lazy loading mode
is set to false (disabled) to avoid behavior changes.
Signed-off-by: Philip Conrad <philip@chariot-chaser.net>
This allows injecting discovery options, such as hooks, or extra
factories, into the runtime. It's useful because when wrapping OPA, you
don't want to re-write the runtime package, you want to use it as-is.
With this, we can still configure a few internals.
Signed-off-by: Stephan Renatus <stephan@styra.com>
I was curious to see how much work this would entail, and it turned out to
be... some :) Particularly porting some of the features exposed as settings
by gorilla mux, like removing trailing slashes, or escaping `/` in matched
paths.
This change is breaking by necessity, as some public functions previously
accepted arguments straight from the mux library. I don't really see any
way around that if we want to get rid of the dependency. I don't think
that too many external projects use code from the server directly though,
so I'm thinking the impact should be minimal? Happy to hear what others
think.
Signed-off-by: Anders Eknert <anders@styra.com>
Following up on #7566, and now applying the more exciting
modernizations. fmt.Appendf was new to me! But especially
the contains checks are so much better IMHO. I have reviewed
all changes myself and did a few manual changes where it
became obvious that things could be improved a little further.
(the modernize analyzer still has some issues running against
OPA, and I have manually worked around those for the time being)
Signed-off-by: Anders Eknert <anders@styra.com>
`TestControlPlaneSpans` could case a race condition, where the discovery plugin is manually triggered before/during server initialization, resulting in the manager config being changed while actively consumed.
Replacing `Runtime.serverInitialized` boolean field with more granular enum type state, to allow test-runtime to hold off on triggering plugins until runtime is actively waiting for plugin ready state.
Currently, manager config writes are guarded by an internal mutex, while config reads are largely unguarded. A broader fix here might be to deprecate the public `plugins.Manager.Context` field, replacing it with a getter that guards the config with an r/w-lock.
Also fixing:
* Possible race condition in telemetry reporter by using r/w-mutex guarded compiler getter instead of direct field access
* AWS signing tests where signing randomly failed because of too small mock random value used in test
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Brace yourselves! For there are many touched files here. No changes
in semantics however.
Spent a long time trying out the various optional rules gocritic
provides, and settled for a few of them. There are more I really
like, but that would take many hours to address across the codebase.
Perhaps others find gocritic too pedantic? If so, we can merge the
fixes without enabling the rule.
Signed-off-by: Anders Eknert <anders@styra.com>
And update code to conform to the rule.
- Replace unnecessary fmt.Sprintf with string concatenation
- Replace fmt.Sprint with more efficient strconv.Itoa
- Replace static fmt.Errorf calls with more efficient errors.New
Thanks @srenatus for pushing me down this rabbit hole!
Signed-off-by: Anders Eknert <anders@styra.com>
This changes updates the docs and all the policy examples in them to
be OPA v1.0-compliant. It also binds the OPA server to `localhost`
interface by default per OPA v1.0 specs.
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
Co-authored-by: Charlie Egan <charlie@styra.com>
When an OPA instance runs for a long time, it seems odd to send version reports
every hour. I think it's unlikely that someone watches the logs at that point.
So this change makes OPA report every 6 hours (plus a random time between 0 and
60 minutes), after it has reported hourly (+spray) for 6 times.
Signed-off-by: Stephan Renatus <stephan@styra.com>
All packages, except for `cmd` and `internal`, have been moved into a new `v1` root package.
Old packages are kept for backwards-compatibility reasons. All contained code is replaced with simple type aliases and proxy functions to `v1` implementations.
Old packages default to the Rego v0 syntax, new `v1` packages default to the Rego v1 syntax.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Sebastian Spaink