To improve plugin and bundle monitoring, new metrics related to bundle-activation
are exported via the prometheus endpoint of the OPA service.
Signed-off-by: rafael otero reinert <rafaelreinert@gmail.com>
- Plugins can now access a channel which receives a message when the OPA
server is fully initialized and ready to receive traffic.
- Added ServerInitialized() and ServerInitializedChannel() to plugin
manager.
- Runtime now calls ServerInitialized() when server listeners are
initialized.
Fixes#3701
Signed-off-by: Grant Shively <gshively@godaddy.com>
This commit fixes the console loggers so that messages are emitted
regardless of the debug log level. The problem was that in 3fcc875 we
updated the plugins to use a console logger obtained from the plugin
manager as opposed to a global logger instantiated in the plugins
package--the console logger obtained from the plugin manager was
instantiated in the runtime package by calling
logging.NewStandardLogger. Unfortunately, logging.NewStandardLogger
does not create a new logger--it returns the global logrus
logger.
This commit fixes the issue by deprecating logging.NewStandardLogger
and introducing two new functions in the logging package:
* logging.Get() - this replaces the old logging.NewStandardLogger
function--this function should be called to obtain the debug logger
used throughout OPA.
* logging.New() - this actually returns a new logger that can be
configured independently from the debug logger used throughout
OPA.
The runtime and sdk packages have been updated to call logging.New()
to obtain console loggers and the rest of the codebase has been
updated to call logging.Get() in place of logging.NewStandardLogger().
Fixes#3654
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
golint is deprecated. The author of the code no longer supports the
codebase. golangci-lint is faster than golint, and is in use by other
opa repositories (e.g. Gatekeeper).
This commit changes tools.go to reference golangci (so it ends up in
vendor) and modifies check-lint to use golangci instead.
Breaking API Changes:
- plugins/rest/rest.go: Fix typo "AllowInsureTLS" -> "AllowInsecureTLS"
- storage/errors.go: Removed unused IndexingNotSupportedErr
Signed-off-by: Will Beason <willbeason@google.com>
* runtime_test: avoid race condition
This had been flagged by our nightly race deteector run. Now, we'll
wait for the server to have stopped before checking its log output.
* plugins: avoid races, bump github.com/sirupsen/logrus
To fix that other one, I've first tried updating logrus (there was a
mention of fixed races in the changelog), but to no avail. Setting up
the hook before any plugin would log from that test resolved the issue.
No harm in updating logrus, though, let's keep that: 1.6.0 -> 1.8.1
* plugins/bundle: fix race
Golang for-range loops need special care when using a reference to the
second variable (v in `for k, v := range m`). We had been copying the
value of m[k], which is a pointer to Status, we had not been -- as was
intended -- copying the values of the struct that the pointer had been
pointing to.
Tests needed to be adapted for this, the s4 update will NOT contain
any bundle-activation-related metrics, as no bundle was activated, and
its status is a fresh copy.
* workflow: add race detector to PR checks
When run from nightly, we use ubuntu-latest; whereas the other checks
in the pull-request workflow use ubuntu-18.04.
I don't think it matters at all for the race detector, since that one
runs only from another docker container, using the golang image.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This commit moves the logging interface and implementations out of the
sdk package into the logging package.
This commit also updates the status and decision log plugins to use a
logger obtained from the plugin manager instead of going to the global
console logger in the plugins package. The latter change will be
important for SDK consumers. This change is backwards incompatible but
it's unlikely that anyone is relying on that export. The test for
console logger independence has also been moved into the plugins
package (from the status package.)
Fixes#3275
Co-authored-by: Torin Sandall <torinsandall@gmail.com>
Co-authored-by: Anders Eknert <anders@eknert.com>
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Signed-off-by: Anders Eknert <anders@eknert.com>
InterQueryBuiltinCacheConfig now responds to the plugin manager's reconfigure event, which allows cache config to exist in discovery config. Previously, cache config would be ignored if it was only declared in discovery config.
Related to #2978.
Signed-off-by: Grant Shively <gshively@godaddy.com>
The WARN state can be used to signal admins that a plugin is in a
potentially dangerous or degraded state. The optional message may be
used to provide context about the warning.
Fixes#2932
Signed-off-by: Grant Shively <gshively@godaddy.com>
Plugins that implement the HTTPAuthPlugin can be used with a new
credentials options under services:
```
services:
my_service:
credentials:
plugin: my_plugin
plugins:
my_plugin: {}
```
Fixes#2758
Signed-off-by: Grant Shively <gshively@godaddy.com>
This commit adds a new inter-query cache that built-in
functions can use to cache responses across queries.
The OPA config includes a new "caching" field that can be used
to set the size of the cache. By default there is no limit.
This change also updates `http.send` to optionally utilize the
inter-query cache.
Fixes#1753
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
This commit refactors the load/store/compile implementation that used
to live inside the runtime package. Specifically:
* Move init-time file loading logic into separate internal package
(initload) along with store/compile logic. Add tests around
load/store/compile that don't require the entire Runtime object.
This also avoids duplication of the "version overwriting" logic.
* Move store/compile calls into the manager. This avoids the need for
two compile operations on startup.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This defines a new status API on the plugins.Manager for plugins
to be able to update their status.
Signed-off-by: Patrick East <east.patrick@gmail.com>
These changes refactor the discovery implementation a bit to improve
test coverage and remove duplication of common logic shared with the
bundle plugin.
Specifically, the downloading logic has been moved into a separate
package that is shared by bundle and discovery. Second, test coverage in
the discovery implementation is increased from ~15% to ~85%.
These changes also include a few functional improvements:
- The default decision paths can be updated dynamically
- The decision logger can be enabled dynamically
- Discovery downloading errors are reported in status updates
- Discovery bundle is evaluated with all runtime params
- Custom plugins can be created dynamically
- Status updates include both discovery and bundle status
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously OPA configuration for bundle downloading, status reporting, etc. had to be supplied in a configuration file on startup. With these changes, OPA can be configured to download a bundle that generates the OPA configuration. This allows OPA to boot with minimal configuration and dynamically update that configuration on-the-fly making it much easier to manage large deployments of OPAs for different use cases within the same system.
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
Stephan Renatus