External rule sources let wrapping projects inject rules at evaluation
time instead of compile time. The compiler marks external packages in
the rule tree but doesn't index them. When topdown hits an external
node, it calls Lookup to get rules, compiles them on the fly with a
scoped compiler, grafts the result into the tree, and caches it for the
duration of the evaluation.
Sources can be isolated (default, no access to surrounding policy) or
non-isolated (can reference static rules and other external sources).
The ExternalRuleIndexCloser interface handles cleanup after evaluation.
Precompiled rules can skip compiler stages via SkippedStages to avoid
redundant work.
This includes:
* hooks: add BundlePreActivate hook This one is handy when registering
external sources.
* topdown: catch `ir == nil` rule index result
This wouldn't ordinarily happen: the compiler is checking refs before.
But in our use case, the SP rules may be configured to be able to reach
into the surrounding Rego (non-isolated mode). If that happens, the IR
lookup may indeed end up as `nil, nil`.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This PR brings support for pluggable logging implementations via the logger plugin interface, which is based on Go's standard log/slog.Handler interface. This allows any slog.Handler implementation to be used as a logger plugin. Loggers can be referenced via the server.logger_plugin configuration option; and can also be used for decision logs. OPA includes a built-in file logger plugin (file_logger) that writes structured JSON logs with rotation support using lumberjack. Users can also implement and register custom logger plugins when building OPA.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
* plugins/rest: cache *http.Client and auth plugin
This will require further changes to cert TLS and token auth methods to
stay compatible with the previous behaviour.
* plugins/rest: configurable re-read interval for TLS cert+key
Defaulting to re-reading all the time, more or less like we did before.
(I write "more or less" because we now do it in `GetClientCertificate()`.)
* plugins/rest: document change (code comments, CHANGELOG)
* plugins/rest: set minimum TLS version where `&tls.Config{}` is used
* plugins/rest: ensure min TLS version and ciphersuites are used
...as configured with the server.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This introduces a new trigger mode for the decision log plugin:
decision_logs.reporting.trigger=immediate
The immediate trigger mode will upload events as soon as enough events are received to hit the configured upload limit. If not enough events are received within the configured min-max delay, the events received so far are flushed and uploaded.
Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
* runtime: Correct naming of version checking code
Rename telemetry functionality to version checking to accurately reflect
current behavior following
https://github.com/open-policy-agent/opa/pull/7756.
The system only checks GitHub releases for version updates without sending
any data about the OPA instance and so the privacy docs have been updated too.
Signed-off-by: Charlie Egan <charlie_egan@apple.com>
* Make WithTelemetryGatherers a no-op
Deprecate WithTelemetryGatherers since telemetry gathering has been removed.
The function now returns a no-op to maintain API compatibility without
breaking existing code that might uses it.
Signed-off-by: Charlie Egan <charlie_egan@apple.com>
---------
Signed-off-by: Charlie Egan <charlie_egan@apple.com>
* v1/plugins: Address race in config access
I ran into this race condition on another PR:
https://github.com/open-policy-agent/opa/actions/runs/16655603110/job/47139789057
I have tried to make all manager.Config access thread-safe by adding new
getters for used values. GetConfig is regrettably based on a JSON
roundtrip deep copy of the config. This us used in tests (fine) but also
in the discovery plugin:
https://github.com/open-policy-agent/opa/blob/2d014a89bbbc307d7204817220146ffae992e838/v1/plugins/discovery/discovery.go#L122
getPluginSet is very tightly coupled to the manager.Config and because
of it's dependencies on status and the other plugins packages, it's hard
to break out.
So, for now, I think this is an improvement and worth getting a second
opinion on before more refactoring.
Signed-off-by: Charlie Egan <charlie@styra.com>
* v1/config: Use add Clone to config
This makes the use of the manager's config more thread-safe and
consistent without more API changes.
Signed-off-by: Charlie Egan <charlie@styra.com>
* topdown: Add clone() funcs for config structs
NamedValueCacheConfig.Clone, InterQueryBuiltinValueCacheConfig.Clone and
InterQueryBuiltinCacheConfig.Clone have been added.
All Clone methods return a deep copy of the struct. This is tested for
missed new fields using PopulateAllFields, a generic function that
stuffs structs with values for all fields.
Signed-off-by: Charlie Egan <charlie@styra.com>
* plugins: Clone new config
Signed-off-by: Charlie Egan <charlie@styra.com>
---------
Signed-off-by: Charlie Egan <charlie@styra.com>
This commit provides an extension mechanism for the server authorizer,
allowing plugins and other server extensions to inform the authorizer
about the methods and paths where it should expect and parse request
bodies.
Signed-off-by: Philip Conrad <philip@chariot-chaser.net>
Before we had introduced `http.ServeMux` as "the router", we had been
using github.com/gorilla/mux. Using the latter, it was possible to
inject middlewares using the mux's `.Use()` method. This mechanism
allowed global middlewares to be injected from `runtime.Params`, for
example.
With `http.ServeMux`, that's no longer possible. However, it was never
an intentionally supported feature in the first place.
So this commit introduces HTTP handler middlewares as extension points.
It's modelled after `(*plugins.Manager).ExtraRoute()`.
Signed-off-by: Stephan Renatus <stephan@styra.com>
This commit adds support for changing out how bundle storage and
activation work. To allow swapping out bundle activation, two new
`bundle` package functions are provided:
- `RegisterActivator`: Registers a bundle.Activator with a string ID.
- `RegisterDefaultBundleActivator`: Sets the default bundle.Activator to
use by ID.
Behind the scenes, a few new `bundle` package variables are used to
track what bundle activators are available, and which is the preferred
default.
This system allows registering many activators, and allows choosing the
bundle activator to use at activation time. The activator to use is
decided in the following order:
- `(bundle.ActivateOpts).Plugin` is used when non-nil.
- `bundle.bundleExtActivator` is used when an ID was set with
`RegisterDefaultBundleActivator`.
- The default/original bundle activator is used if no other selection
was made.
To support swapping out bundle storage (useful when testing new bundle
designs), a new `bundle` package function is provided:
- `RegisterStoreFunc`: Sets the function to use for creating bundle
storage.
These two features together allow swapping out most of the bundle
activation flow, without requiring deep modification of the `bundle`
package. Lazy bundle loading mode is also enabled across many CLI
commands and other bundle loading points now when a non-default bundle
activator is set.
Signed-off-by: Philip Conrad <philip@chariot-chaser.net>
Co-authored-by: Ashutosh Narkar <anarkar4387@gmail.com>
This way, the extra handler functions are still covered by prometheus
metrics and opentelemetry spans.
The previous method of directly registering routes with the router
bypassed the server's handler wrapping.
Signed-off-by: Stephan Renatus <stephan@styra.com>
I was curious to see how much work this would entail, and it turned out to
be... some :) Particularly porting some of the features exposed as settings
by gorilla mux, like removing trailing slashes, or escaping `/` in matched
paths.
This change is breaking by necessity, as some public functions previously
accepted arguments straight from the mux library. I don't really see any
way around that if we want to get rid of the dependency. I don't think
that too many external projects use code from the server directly though,
so I'm thinking the impact should be minimal? Happy to hear what others
think.
Signed-off-by: Anders Eknert <anders@styra.com>
Following up on #7566, and now applying the more exciting
modernizations. fmt.Appendf was new to me! But especially
the contains checks are so much better IMHO. I have reviewed
all changes myself and did a few manual changes where it
became obvious that things could be improved a little further.
(the modernize analyzer still has some issues running against
OPA, and I have manually worked around those for the time being)
Signed-off-by: Anders Eknert <anders@styra.com>
`TestControlPlaneSpans` could case a race condition, where the discovery plugin is manually triggered before/during server initialization, resulting in the manager config being changed while actively consumed.
Replacing `Runtime.serverInitialized` boolean field with more granular enum type state, to allow test-runtime to hold off on triggering plugins until runtime is actively waiting for plugin ready state.
Currently, manager config writes are guarded by an internal mutex, while config reads are largely unguarded. A broader fix here might be to deprecate the public `plugins.Manager.Context` field, replacing it with a getter that guards the config with an r/w-lock.
Also fixing:
* Possible race condition in telemetry reporter by using r/w-mutex guarded compiler getter instead of direct field access
* AWS signing tests where signing randomly failed because of too small mock random value used in test
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Brace yourselves! For there are many touched files here. No changes
in semantics however.
Spent a long time trying out the various optional rules gocritic
provides, and settled for a few of them. There are more I really
like, but that would take many hours to address across the codebase.
Perhaps others find gocritic too pedantic? If so, we can merge the
fixes without enabling the rule.
Signed-off-by: Anders Eknert <anders@styra.com>
where the bundle has been manually constructed containing v0 Rego modules and
no `rego_version`/`file_rego_versions` fields are declared in the bundle manifest.
This affects bundle deactivation in the bundle store lifecycle used when for
`opa run` in server mode (`-s`).
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
I pushed most of these fixed previously but these ones required
more work. Probably more work that it was worth, lol, but now the
work is done... and at least we can add this to the list of enabled
checks.
Fixes#7238
Signed-off-by: Anders Eknert <anders@styra.com>
All packages, except for `cmd` and `internal`, have been moved into a new `v1` root package.
Old packages are kept for backwards-compatibility reasons. All contained code is replaced with simple type aliases and proxy functions to `v1` implementations.
Old packages default to the Rego v0 syntax, new `v1` packages default to the Rego v1 syntax.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Stephan Renatus