Watching files only works in situations where standard files are in
use. In k8s, configmaps are mounted via a set of symlinks. In those
situations, you will only get file events when watching the directory
containing the symlink.
Fixes#2588
Signed-off-by: Drew Wells <drew.wells00@gmail.com>
These changes add support for digital signatures for policy bundles which
can be used to verify their authenticity.
Bundle signature verification involves the following steps:
* Verify the JWT signature
* Verify the files in the JWT payload exist in the bundle
* Verify the file content of the files in bundle match with those in the payload
This commit adds a new `sign` command to generate a digital signature for policy bundles.
For more details, run "opa sign --help"
The signatures generated by the 'sign' command can be verified by the
'build' command. The 'build' command can also sign the bundle it generates.
The 'run' command can verify a signed bundle or skip verification altogether.
OPA 'sign', 'build' and 'run' can be used to
sign/verify bundles in bundle mode (--bundle) mode only. Verification
can be also be performed when bundle downloading is enabled.
Fixes: #1757
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
This commit does a few things:
* Extend the low-level file loader and callers to ensure ModuleFile
objects have a URL field. This means we can tell if two ModuleFile
objects refer to the same source file. Previously we had to keep track
of bundle names to do this. This commit does not touch the bundle
activation logic which does this already--we can revisit that in the
future.
* Update the bundle writer to use the URL field by default. This
allows us to write merged bundles correctly.
* Add a merge function that will be used by the new build command to
combine multiple bundles.
* Add a helper function for checking if roots contain a path. In the
process refactor the overlap helper to use the same logic. Also add a
helper function to ensure roots exist in the bundle manifest.
* Add a deepcopy function for bundles. This is useful if mutating
bundles and there is a need to revert.
* Format bundle contents on write by default. Callers of the old
Write function will not be affected (it disables formatting.)
* Format the overlap error to quote roots in case they are empty.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
The documentation is pretty clear that a module must at least contain
a package, so it is safe to say that an empty file isn't valid.
Previously the helper would just return a nil module and nil error, it
will now return an error.
Fixes: #2054
Signed-off-by: Patrick East <east.patrick@gmail.com>
When reading a bundle from a directory we should be using the full
system path for the module files. This helps greatly with reducing
complexity of trying to read error messages. It also makes the
integration with tools like VSCode work better as they can provide
links from the console output file paths to the file in question.
This will not affect bundles loaded from tarballs.
Fixes: #1796
Signed-off-by: Patrick East <east.patrick@gmail.com>
This corrects the missing time in rego_module_parse timers as we now
have metrics collecting info as we parse *.rego files from file
loaders and from bundles as they are unpacked.
It also adds in a timer for the data files that are loaded through
similar mechanisms.
Signed-off-by: Patrick East <east.patrick@gmail.com>
Previously we had it in an internal package but used by a public API,
which basically means it can't actually be used outside of OPA.
Initial thinking was that this was an OK situation, but by popular
demand we are making it available to everyone so OPA as a lib users
can use the bundle loading API's.
Fixes: #1840
Signed-off-by: Patrick East <east.patrick@gmail.com>
Previously if the errors passed into the presentation Output were not
structured w/ JSON tags for marshaling the error would be an empty
string.
This changes to wrap the errors with a struct in cases where they
would otherwise not be formatted. We do this by forcing every error
into a structure and translating known error types into it.
Fixes: #1726Fixes: #1724
Signed-off-by: Patrick East <east.patrick@gmail.com>
Users should be able to pass file:// URLs to any of the
sub-commands. In 3be55ed6 the eval and fmt sub-commands were not
updated to accept file:// URLs for the input file and normal paths
(respectively).
This commit just moves the unexported cleanFileURL function from the
loader package into it's own internal package so that it can be shared
in OPA.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This adds a new API to load a bundle from a path which can be either
a tarball file or a directory to load as a bundle.
Signed-off-by: Patrick East <east.patrick@gmail.com>
The file loader splits paths on the first colon character and uses the
left-hand side for the prefix to root the document at under data. On
windows this is problematic because of drive lettesr (e.g., C:\X\Y\Z
is interpreted as load file at \X\Y\Z under data.C.
This change updates the loader to accept file:// URLs. This way
callers can unambiguously specify filenames that contain colon
characters. For now this will mainly be used by VS Code and other
programmatic callers. In future we can support other schemes (e.g., http).
Fixes#1505
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously, when OPA activated a bundle, it would erase ALL existing
policy and data that had been cached. This meant that the bundles and
components like kube-mgmt were mutually exclusive (because the
bundles would overwrite the other component's policy and data.)
With these changes, bundles can include a set of roots that scope the
bundle. When the bundle activates, only policy and data under those
roots are erased and overwitten.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously OPA would only load JSON/YAML/Rego files off the command
line. With these changes, OPA will load .tar.gz files and interpret them
as bundles. This is useful if you want to test your bundles locally with
OPA without running OPA as a server, configuring it to pull down the
bundle, etc.
Also, update docs to mention that data files MUST be named data.json.
Fixes#870Fixes#873
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
I don't know why I ended up looking into this, but I've run
golangci-lint run --disable-all -E deadcode
on the code base and removed everything that came up :)
Signed-off-by: Stephan Renatus <srenatus@chef.io>
These changes add filtering support to the loader. Users of the loader
package can provide a function that will filter the files included in
the loader.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
The file loader logic was unnecessarily duplicated across the top-level
file case and the recursive file case. These changes refactor the
implementation so that the logic is in one place and the file walker has
access to the depth which could be used in the future to place a limit
on recursion depth.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
The check subcommand was not formatting parse errors as JSON if
requested. Also, since the loader package returns a set of errors, parse
errors are unpacked to avoid double nesting.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously, policies and data had to be pushed into OPA via the REST API
or loaded via command line arguments at startup.
With these changes, OPA can now be configured to pull down bundles of
policy and data from remote HTTP servers. When a bundle is downloaded
successfully, the policies and data are loaded out of the bundle file
and inserted into storage.
Previously, the loader would use directory names as top-level keys when
paths referred to directories. This meant that identical queries against
policies/data in differently named directories would return different
answers.
Now, the loader ignores the first directory name when recursing on
paths. Unfortunately this is not backwards compatible. Scripts and
workflows can be adapted as follows:
Before: opa test *
After: opa test .
Before: opa test /some/path/to/dir/*
After: opa test /some/path/to/dir
The same goes for opa run.
Previously, the loader would read the input file for each guess. For
non-file inputs (e.g., process substitution) this would fail. Now you
can pipe command output into OPA over the command line!
$ opa run <(echo '{"foo":"bar"}') -e 'data.foo'
"bar"
The file loading logic implemented in the runtime package is generally
useful within OPA. These changes factor the file loading into a separate
package that can be reused without taking a dependency on the runtime.
Drew Wells