This change brings in support for multiple bundles to be downloaded
and activated OPA.
This is enabled by using the new config option `bundles` to define
the bundles, and deprecates the older `bundle` option.
The new `bundles` keyword and structure is propagated through to the
decision logs, status API, provenance, stored manifests, etc. Check
out the doc changes for all the updated structures.
That being said any existing configuration using `bundle` will *not*
see the new structure, everything is intended to be backwards
compatible (almost to a fault).
Fixes: #721
Signed-off-by: Patrick East <east.patrick@gmail.com>
These changes update the manager and bundle plugin to avoid parsing
and compiling modules during the manager's trigger callback. Since
policy queries are blocked while triggers execute, it's adavantageous
to cache the compiler that is obtained during bundle activation and
reduce the work done in the trigger callback.
Also, as part of these changes, the bundle plugin incorporates
remaining modules when it recompiles. This ensures that remaining
modules are correct.
Fixes#1515
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously we had a bunch of ad-hoc code to store the revision and
roots internally. These changes just add a proper package to deal with
the reading and writing of the revision and roots.
Note, the bundle package itself is still hardcoding the manifest path
but we can fix that in a subsequent PR.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously, when OPA activated a bundle, it would erase ALL existing
policy and data that had been cached. This meant that the bundles and
components like kube-mgmt were mutually exclusive (because the
bundles would overwrite the other component's policy and data.)
With these changes, bundles can include a set of roots that scope the
bundle. When the bundle activates, only policy and data under those
roots are erased and overwitten.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously there were no checks in place to ensure that base and
virtual documents do not overlap. As a result, if users loaded raw
JSON and rules into OPA that overlapped, the evaluation results were
not well defined. With these changes, we can detect the overlap and
reject updates (to policies or data) that would cause inconsistent
results.
Fixes#1207
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This change allows to set the Prefix to the path of the URI.
Until this commit the prefix statically set as "bundles" and was concatenated by the bundle plugin.
Signed-off-by: liorlevtov <liorl@luminate.io>
These changes refactor the discovery implementation a bit to improve
test coverage and remove duplication of common logic shared with the
bundle plugin.
Specifically, the downloading logic has been moved into a separate
package that is shared by bundle and discovery. Second, test coverage in
the discovery implementation is increased from ~15% to ~85%.
These changes also include a few functional improvements:
- The default decision paths can be updated dynamically
- The decision logger can be enabled dynamically
- Discovery downloading errors are reported in status updates
- Discovery bundle is evaluated with all runtime params
- Custom plugins can be created dynamically
- Status updates include both discovery and bundle status
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously OPA configuration for bundle downloading, status reporting, etc. had to be supplied in a configuration file on startup. With these changes, OPA can be configured to download a bundle that generates the OPA configuration. This allows OPA to boot with minimal configuration and dynamically update that configuration on-the-fly making it much easier to manage large deployments of OPAs for different use cases within the same system.
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
I don't know why I ended up looking into this, but I've run
golangci-lint run --disable-all -E deadcode
on the code base and removed everything that came up :)
Signed-off-by: Stephan Renatus <srenatus@chef.io>
Previously, the bundle plugin was only updating the status struct if:
* a download or activation error occurred.
* a new bundle was successfully activated.
As a result, if a transient bundle download/activation error occcured,
OPA would never report that the error condition had been cleared in the
event that the service began serving HTTP 304s.
These changes update the bundle plugin to always set the error status
regardles of whether an error occurred or a new bundle was activated.
Also, refactor the bundle plugin to set all of the activation state
(timestamp, revision, etag) in one place.
Fixes#794
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously, the bundle plugin did not expose a mechanism for callers to
inspect the status of bundle downloads. With these changes, callers can
register for notifications when bundle updates are performed. This will
allow OPA to report status to remote HTTP servers.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously, policies and data had to be pushed into OPA via the REST API
or loaded via command line arguments at startup.
With these changes, OPA can now be configured to pull down bundles of
policy and data from remote HTTP servers. When a bundle is downloaded
successfully, the policies and data are loaded out of the bundle file
and inserted into storage.
Patrick East