This change allows users that build their own executable or "spin" of
OPA to give it a name, and have it reference itself properly in help
texts.
It's a vanity thing, but I think some people would appreciate it, hat
tip to the international association of pedants.
Signed-off-by: Stephan Renatus <stephan@styra.com>
Co-authored-by: kevinstyra <83973046+kevinstyra@users.noreply.github.com>
`os.Exit` immediately exits the program and doesn't run defer functions.
This can be problematic as any command.OnFinalize routines and any logic
after the command.Execute won't be run.
Also suppress all RunE cobra error and usage messages. These would be
printed twice otherwise.
Signed-off-by: Stephan Renatus <stephan@styra.com>
Co-authored-by: Kevin St. Pierre <kevin@styra.com>
This commit adds support for changing out how bundle storage and
activation work. To allow swapping out bundle activation, two new
`bundle` package functions are provided:
- `RegisterActivator`: Registers a bundle.Activator with a string ID.
- `RegisterDefaultBundleActivator`: Sets the default bundle.Activator to
use by ID.
Behind the scenes, a few new `bundle` package variables are used to
track what bundle activators are available, and which is the preferred
default.
This system allows registering many activators, and allows choosing the
bundle activator to use at activation time. The activator to use is
decided in the following order:
- `(bundle.ActivateOpts).Plugin` is used when non-nil.
- `bundle.bundleExtActivator` is used when an ID was set with
`RegisterDefaultBundleActivator`.
- The default/original bundle activator is used if no other selection
was made.
To support swapping out bundle storage (useful when testing new bundle
designs), a new `bundle` package function is provided:
- `RegisterStoreFunc`: Sets the function to use for creating bundle
storage.
These two features together allow swapping out most of the bundle
activation flow, without requiring deep modification of the `bundle`
package. Lazy bundle loading mode is also enabled across many CLI
commands and other bundle loading points now when a non-default bundle
activator is set.
Signed-off-by: Philip Conrad <philip@chariot-chaser.net>
Co-authored-by: Ashutosh Narkar <anarkar4387@gmail.com>
A tiny first step to have more tooling correctly report
virtual and base document conflicts, as detailed in #7694.
This PR fixes the `opa check` command to report conflicts
of this type when the `-b`/`--bundle` flag is provided. The
bundle flag is required as without that, `opa check` should
only verify policies and not load data at all.
While I was in the `cmd` directory, I got annoyed with how
many of these commands store the same constants for their
`--format` flag, so I decided to fix that too, even if it
wasn't related to what I originally planned to do. I hope
it's not too distracting.
Signed-off-by: Anders Eknert <anders@styra.com>
Following up on #7566, and now applying the more exciting
modernizations. fmt.Appendf was new to me! But especially
the contains checks are so much better IMHO. I have reviewed
all changes myself and did a few manual changes where it
became obvious that things could be improved a little further.
(the modernize analyzer still has some issues running against
OPA, and I have manually worked around those for the time being)
Signed-off-by: Anders Eknert <anders@styra.com>
The new command is based on generating JSON for docusaurus consumption
rather than markdown. This is less error prone as manipulation of
markdown is better contained.
Signed-off-by: Charlie Egan <charlie@styra.com>
And update code to conform to the rule.
- Replace unnecessary fmt.Sprintf with string concatenation
- Replace fmt.Sprint with more efficient strconv.Itoa
- Replace static fmt.Errorf calls with more efficient errors.New
Thanks @srenatus for pushing me down this rabbit hole!
Signed-off-by: Anders Eknert <anders@styra.com>
To make OPA behave as v0.x post v1.0 release.
If used simultaneously with `--v1-compatible` flag, the `--v0-compatible` flag takes precedence.
Also, future-proofing `cmd` package tests for 1.0.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* ast+cmd+rego: Adding `--rego-v1` flag to `opa eval`
Fixes: #6463
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* Adding `--rego-v1` flag to `opa build`
Fixes: #6463
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* Formatting PE support modules to comply with rego-v1 when required
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* Removing rego.v1 import when formatting for rego-v1 (not rego-v0-compat-v1)
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* touch up
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* Fixing linting issues
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* Consolidating `Bundle.FormatModules()` and `Bundle.FormatModulesForRegoVersion()`
Suggested by @ashutosh-narkar
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* Adding descriptions to `RegoVersion`
Requested by @ashutosh-narkar
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* Using `--v1-compatible` flag instead of `--rego-v1`
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* Updating docs
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* Reintroducing `ParserOptions.RegoV1Compatible` to avoid breaking change
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* cmd & tester
Adding `--v1-compatible` flag to `opa test`
Fixes: #6463
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* Adding `--v1-compatible` flag to `opa fmt`
Fixes: #6463
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* Adding `--v1-compatible` flag to `opa check`
Fixes: #6463
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* Making linter happy
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* Review modifications suggested by @ashutosh-narkar
* Changing `ParserOptions.RegoV1Compatible` take precedence over `ParserOptions.RegoVersion`
* Fixing comment in test
* Updating `fmt --rego-v1` CLI description
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* Review modifications suggested by @ashutosh-narkar
* Changing `ParserOptions.RegoV1Compatible` take precedence over `ParserOptions.RegoVersion`
* Fixing comment in test
* Updating `fmt --rego-v1` CLI description
* Adding back `Opts.RegoV1` and deprecating.
* Making `Opts.RegoV1` take precedence over `Opts.RegoVersion`
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* Review modifications suggested by @ashutosh-narkar
* Changing `ParserOptions.RegoV1Compatible` take precedence over `ParserOptions.RegoVersion`
* Fixing comment in test
* Updating `fmt --rego-v1` CLI description
* Adding back `Opts.RegoV1` and deprecating.
* Making `Opts.RegoV1` take precedence over `Opts.RegoVersion`
* `TestPartialWitRegoV1` -> `TestPartialWithRegoV1`
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* Review modifications suggested by @ashutosh-narkar
* Changing `ParserOptions.RegoV1Compatible` take precedence over `ParserOptions.RegoVersion`
* Fixing comment in test
* Updating `fmt --rego-v1` CLI description
* Adding back `Opts.RegoV1` and deprecating.
* Making `Opts.RegoV1` take precedence over `Opts.RegoVersion`
* `TestPartialWitRegoV1` -> `TestPartialWithRegoV1`
* removing `Println` in test
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* Review modifications suggested by @ashutosh-narkar
* Changing `ParserOptions.RegoV1Compatible` take precedence over `ParserOptions.RegoVersion`
* Fixing comment in test
* Updating `fmt --rego-v1` CLI description
* Adding back `Opts.RegoV1` and deprecating.
* Making `Opts.RegoV1` take precedence over `Opts.RegoVersion`
* `TestPartialWitRegoV1` -> `TestPartialWithRegoV1`
* removing `Println` in test
* Updating docs with per-command behavioural descriptions for `--v1-compatible`.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
---------
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* ast: Enforce inlined schemas without --schemas flag
Enabled for commands:
* eval
* check
* test
Always parse annotations.
Always use `schemas` annotations with inlined schemas for type checking.
Ignore `schemas` annotations with schema refs if no `--schema` flag was provided.
Fixes: #5506
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Fixing an issue where if a `rego.metadata.*` function is called in a rule, any present `schemas` annotation will be used to inform type checking, even if no `--schema` flag was used for the command.
Only commands that takes the `--schema` flag can now use the `schemas` annotation to inform type checking; i.e. `eval` and `check`.
Fixes: 5430
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Before, the capabilities were plumbled through in most places:
1. checking which builtins exist
2. passed along to the optimizer
3. passed along to the planner
But they hadn't been passed along to the file loader. As such, it could not
pass the caps along to the parser either. This is now done, but adding a new
method to the FileLoader interface.
Fixes#5323.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This adds a new top-level key to the capabilities structure, `allow_net`.
It currently is only used for restricting the typechecker's ability to fetch
remote refs in JSON schemas, but could be used more widely in the future.
It works like this:
- If it's not present, any host can be contacted
- If it's present, the items will be the hosts or IP addresses that may be
contacted; anything not in the list is prohibited.
- As a consequence, If it's present and empty (`[]`), no host can be contacted
Introducing a package-level var to gojsonschema isn't the prettiest solution,
but since we want this in an all-or-nothing way right now anyways, it does
the trick. And it's more ergonomic than adding extra parameters all over the
place.
Fixes#3746.
Also:
* move some profiling-related default params into newEvalCommandParams
* replace some errors.Wrap by fmt.Errorf in loader pkg
* remove some != nil handling where it didn't make a difference when
working on the schema set
* reduces indentation in code examples in `opa eval -h` and `opa check -h`
by replacing tabs by four spaces.
* ast: allow testing with remote refs without networking
It would be nice to ensure that the remote refs feature actually works,
without introducing a network dependency into our tests.
This commit adds the kube 1.14 definitions into ast/testdata, and uses
that from a httptest.Server instance in the unit tests.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
The build and check commands were retrieving registered builtins too
early during initial execution, which did not give enough time for
custom binaries to register custom builtin functions. This change
sets the initial capabilities flag to nil instead, and then retrieves built-ins
immediately prior to instantiating a new compiler. This prevents an
undefined function rego_type_error during the build and check commands.
Fixes#2738.
This is my commit message
Signed-off-by: Grant Shively <gshively@godaddy.com>
These changes add support for digital signatures for policy bundles which
can be used to verify their authenticity.
Bundle signature verification involves the following steps:
* Verify the JWT signature
* Verify the files in the JWT payload exist in the bundle
* Verify the file content of the files in bundle match with those in the payload
This commit adds a new `sign` command to generate a digital signature for policy bundles.
For more details, run "opa sign --help"
The signatures generated by the 'sign' command can be verified by the
'build' command. The 'build' command can also sign the bundle it generates.
The 'run' command can verify a signed bundle or skip verification altogether.
OPA 'sign', 'build' and 'run' can be used to
sign/verify bundles in bundle mode (--bundle) mode only. Verification
can be also be performed when bundle downloading is enabled.
Fixes: #1757
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
This adds in a new CLI sub command `opa bench` which will load
and evaluate similar to `opa eval` but will perform benchmark testing
of the query.
There is also a new `--bench` option for `opa test` which will
similarly perform benchmarking, except on the unit tests.
Both use the golang testing frameworks benchmark tooling, and the
output format is compliant with the go benchmark standard when using
the newly added `gobench` output format option.
They both support specifying a `--count` to run the benchmark a
number of times and a `--benchmem` option to report memory statistics.
To help enable using the `opa test` command better with the benchmark
option there is now a `--run`/`-r` option that can be provided to
specify a regex for what test cases should be run. The regex supports
anything that is supported by re2:
https://github.com/google/re2/wiki/Syntax
These changes required updating to Go 1.13 to get the ability to
report custom metrics with the benchmark results
https://golang.org/pkg/testing/#B.ReportMetric To get Netlify on board
we needed to add a `.go-version` file to the root of the repo. This is
now the single source of truth for the OPA golang version.
Fixes: #1424
Signed-off-by: Patrick East <east.patrick@gmail.com>
This corrects the missing time in rego_module_parse timers as we now
have metrics collecting info as we parse *.rego files from file
loaders and from bundles as they are unpacked.
It also adds in a timer for the data files that are loaded through
similar mechanisms.
Signed-off-by: Patrick East <east.patrick@gmail.com>
Previously if the errors passed into the presentation Output were not
structured w/ JSON tags for marshaling the error would be an empty
string.
This changes to wrap the errors with a struct in cases where they
would otherwise not be formatted. We do this by forcing every error
into a structure and translating known error types into it.
Fixes: #1726Fixes: #1724
Signed-off-by: Patrick East <east.patrick@gmail.com>
Previously we would internally reference modules by only their `path`
which was, for data files, the system path but bundles it is relative
to the root of the bundle. In theory data paths and bundle paths could
collide, but the real trouble is caused by multiple bundles. It was
very easy to have two bundles with identical file paths but different
packages and policies defined in them.
Internally we now reference bundle module id's as a combination of the
bundle name (or the file path for the bundle if loaded from CLI) and
the path within the bundle.
This does change the `id` a particular policy will show up at via the
storage ListPolicies and in turn REST API for OPA. This only affects
users that have switched to the `bundles` configuration option, or
that are using the `-b`/`--bundle` CLI options to load bundles. The
older style `bundle` config keyword and loading tarballs from as data
paths are still going to use the older ID.
Fixes: #1725
Signed-off-by: Patrick East <east.patrick@gmail.com>
The option will work similarly to the `test` and `run` commands where
CLI arguments passed in as files or directories are treated as bundles
when the flag is provided.
Signed-off-by: Patrick East <east.patrick@gmail.com>
I don't know why I ended up looking into this, but I've run
golangci-lint run --disable-all -E deadcode
on the code base and removed everything that came up :)
Signed-off-by: Stephan Renatus <srenatus@chef.io>
These changes update the subcommands to support a file/directory name
filter. This allows users to exclude certain files from being loaded.
With these changes users can excldue private directories created by
Kubernetes for volume-mounted ConfigMaps.
As part of this change, update the Kubernetes deployment documentation
to use the new --ignore flag, run OPA as a Deployment instead of as a
ReplicationController, and generally improve the example.
Fixes#782
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
The check subcommand was not formatting parse errors as JSON if
requested. Also, since the loader package returns a set of errors, parse
errors are unpacked to avoid double nesting.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
The file loading logic implemented in the runtime package is generally
useful within OPA. These changes factor the file loading into a separate
package that can be reused without taking a dependency on the runtime.
Sometimes compiling large policies with many errors causes more output
errors than is easily sorted through. The compiler has been updated to
cancel after a configurable number of errors (default no limit), and the
server, repl and check command support options for setting that limit.
kevinstyra