This change brings in support for multiple bundles to be downloaded
and activated OPA.
This is enabled by using the new config option `bundles` to define
the bundles, and deprecates the older `bundle` option.
The new `bundles` keyword and structure is propagated through to the
decision logs, status API, provenance, stored manifests, etc. Check
out the doc changes for all the updated structures.
That being said any existing configuration using `bundle` will *not*
see the new structure, everything is intended to be backwards
compatible (almost to a fault).
Fixes: #721
Signed-off-by: Patrick East <east.patrick@gmail.com>
This will allow others consuming OPA as a library to have easier
access to manage/query the bundle manifests.
Signed-off-by: Patrick East <east.patrick@gmail.com>
These changes simply update the bundle reader to accept data.yaml
files in addition to data.json files. This allows callers to supply
YAML files to OPA without having to convert them into JSON.
Fixes#1471
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
When we parse modules we won’t get an error back if the module contents
is empty, but we do get back a `nil` module value.
In the bundle loader we need to catch this before going further with
trying to load the module as it is not actually a valid module and
will break assumptions made about it.
According to the docs Modules must, at a minimum, have a package
declaration. With that in mind this seems like the right behavior to
enforce on the bundled rego files since we treat them as modules.
Fixes: #1393
Signed-off-by: Patrick East <east.patrick@gmail.com>
Previously, when OPA activated a bundle, it would erase ALL existing
policy and data that had been cached. This meant that the bundles and
components like kube-mgmt were mutually exclusive (because the
bundles would overwrite the other component's policy and data.)
With these changes, bundles can include a set of roots that scope the
bundle. When the bundle activates, only policy and data under those
roots are erased and overwitten.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously, the bundle reader code would only load data.json files
prefixed with / characters. This meant that if the data file was
located under a subdirectory it would load fine in all cases, but if it
was contained at the root, the tar file would have to be written to
include the slash prefix. By default running tar on a directory will not
do this--which meant that data.json files would not be picked up if the
bundle was created using the tar command.
These changes modify the bundle reader to accept data.json files in all
of these cases. The behaviour between opa run . and opa run
bundle.tar.gz is consistent regardless of whether the bundle was written
by OPA or by tar.
Also, since the file extensions are not used outside of the bundle
package, unexport them.
Fixes#1019
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
These chanages introduce a manifest to the bundle files. This way bundle
services can include metadata about the bundle in the file. The initial
use case for bundle data is keeping track of bundle revisions.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously, policies and data had to be pushed into OPA via the REST API
or loaded via command line arguments at startup.
With these changes, OPA can now be configured to pull down bundles of
policy and data from remote HTTP servers. When a bundle is downloaded
successfully, the policies and data are loaded out of the bundle file
and inserted into storage.
Patrick East