Users should be able to pass file:// URLs to any of the
sub-commands. In 3be55ed6 the eval and fmt sub-commands were not
updated to accept file:// URLs for the input file and normal paths
(respectively).
This commit just moves the unexported cleanFileURL function from the
loader package into it's own internal package so that it can be shared
in OPA.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This adds a new API to load a bundle from a path which can be either
a tarball file or a directory to load as a bundle.
Signed-off-by: Patrick East <east.patrick@gmail.com>
The file loader splits paths on the first colon character and uses the
left-hand side for the prefix to root the document at under data. On
windows this is problematic because of drive lettesr (e.g., C:\X\Y\Z
is interpreted as load file at \X\Y\Z under data.C.
This change updates the loader to accept file:// URLs. This way
callers can unambiguously specify filenames that contain colon
characters. For now this will mainly be used by VS Code and other
programmatic callers. In future we can support other schemes (e.g., http).
Fixes#1505
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously OPA would only load JSON/YAML/Rego files off the command
line. With these changes, OPA will load .tar.gz files and interpret them
as bundles. This is useful if you want to test your bundles locally with
OPA without running OPA as a server, configuring it to pull down the
bundle, etc.
Also, update docs to mention that data files MUST be named data.json.
Fixes#870Fixes#873
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
These changes add filtering support to the loader. Users of the loader
package can provide a function that will filter the files included in
the loader.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
The file loader logic was unnecessarily duplicated across the top-level
file case and the recursive file case. These changes refactor the
implementation so that the logic is in one place and the file walker has
access to the depth which could be used in the future to place a limit
on recursion depth.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously, policies and data had to be pushed into OPA via the REST API
or loaded via command line arguments at startup.
With these changes, OPA can now be configured to pull down bundles of
policy and data from remote HTTP servers. When a bundle is downloaded
successfully, the policies and data are loaded out of the bundle file
and inserted into storage.
Previously, the loader would use directory names as top-level keys when
paths referred to directories. This meant that identical queries against
policies/data in differently named directories would return different
answers.
Now, the loader ignores the first directory name when recursing on
paths. Unfortunately this is not backwards compatible. Scripts and
workflows can be adapted as follows:
Before: opa test *
After: opa test .
Before: opa test /some/path/to/dir/*
After: opa test /some/path/to/dir
The same goes for opa run.
Previously, the loader would read the input file for each guess. For
non-file inputs (e.g., process substitution) this would fail. Now you
can pipe command output into OPA over the command line!
$ opa run <(echo '{"foo":"bar"}') -e 'data.foo'
"bar"
The file loading logic implemented in the runtime package is generally
useful within OPA. These changes factor the file loading into a separate
package that can be reused without taking a dependency on the runtime.
Patrick East