This commit adds a new flag to the opa bench command which
allows users to run benchmarks against a running OPA server.
This mode can be used to evaluate the additional overhead the
server is going to introduce.
Co-authored-by: Anders Eknert anders@eknert.com
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
The change enables using --diff and --list together with --fail as
discussed in #4508, for example:
$ opa fmt [path [...]] --list --fail; exit $?
path/to/file-1.rego
path/to/file-2.rego
unexpected diff
2
Previously, the same command returned only the error:
$ opa fmt [path [...]] --list --fail; exit $?
unexpected diff
2
Signed-off-by: David Kuridža <david@kuridza.si>
This package is deprecated, archived, and in maintenance mode, since Go
errors support wrapping natively.
For #2152.
Signed-off-by: Jason Hall <jason@chainguard.dev>
* format: allow ignoreing source locations
* cmd/eval: format disregarding source locations for partial result
Before, we'd see this output:
```
$ opa eval -p -fsource 'time.clock(input.x)==time.clock(input.y)'
# Query 1
time.clock(time.clock(input.x), input.y)
```
Now, we get the proper answer: `time.clock(input.y, time.clock(input.x))`.
Note that it's a _display_ issue; the JSON output of PE has not been affected.
Fixes#4609.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
There is a new command argument "capabilities". With this, it is
possible to print the current capabilities version, show all
capabilities versions & print any capabilities version, without the need
of a file. Moreover, for the other commands which use the --capabilities
flag, it is possible to give only the version number, without specifying
a file. However, there are no breaking changes for those who use the
capabilities file as an input for the flag. Unit tests were also
written, in order to test the new argument and the changes made in ast.
Fixes: #4236
Signed-off-by: IoannisMatzaris <matzarisioannis@gmail.com>
New functions:
* rego.metadata.chain(): returns the chain of metadata, starting from the active rule, going outward
* rego.metadata.rule(): returns the metadata for the active rule
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Some small changes:
* cmd/inspect: inline ast.DropHead() as dropDataPrefix() -- to avoid the extra export in the ast package
* docs/annotations: update related_resources example
* cmd/inspect: unexport PageWidth
* cmd/inspect: make usage text markdown-friendly
This looks better on the website docs, and doesn't matter much when
viewed on the CLI.
* cmd/inspect: add test for pretty annotations output
* cmd/inspect: also render related_resources in pretty mode
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
By passing `-a` to `opa inspect`, it will pretty-print information
about the rules' and packages' annotations.
For all annotations, including related_resources, use `-fjson`.
Fixes: #4387
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
This commit adds a new "plan" target to the compile package and build
command. This targets lets users compile out query plans into
bundles. This way the compiled query plan can be consumed by
transpilers/interpreters outside of OPA.
Fixes#4133
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This change adds linux/arm64 binaries to the release. It also publishes an arm64
container image for all variants (standard, debug, rootless, static) and releases
(dev, edge, latest).
The build and push process uses buildx in order to push the individual
images by digest (i.e. untagged) and reference them in a single, tagged manifest
list. This avoids cluttering Docker Hub's tag list with `<tag>-<arch>` tags.
Fixes#2233
Signed-off-by: Nick Graef <1031317+ngraef@users.noreply.github.com>
This is just a skeleton but the basic functionality is there: run OPA
in a "one shot" mode against a set of input files and print the
results for each.
Fixes#3525
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This follows the same approach as the wasm feature: by default, importers
of
github.com/open-policy-agent/opa/rego
github.com/open-policy-agent/opa/topdown
will not get a transitive dependency on the otel libraries.
In terms of functionality, nothing changes for the server and runtime.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This commit implements tracing using the net/http automatic
instrumentation wrappers on the server and topdown/http packages.
Fixes#1469
Signed-off-by: Rien Valkenaers <rien.valkenaers@gmail.com>
This adds a new flag to `opa run`, intended for server usage with HTTPS listeners:
`--tls-cert-refresh-period`. If used with a positive duration, such as "5m" (5 minutes),
"24h", etc, the server will track the certificate and key files' contents. When their
content changes, the certificates will be reloaded.
On an error in reloading, it will log (info) the error and try again in the next round.
Fixes#2500.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Inspect previously pretty printed only when -f pretty was specified. This change ensures that inspect pretty prints by default, and a user can specify --format=json to avoid relying on implicit default behaviour.
Fixes: #4021
Signed-off-by: Emily Tao
<tao.emily@yahoo.ca>
Signed-off-by: Emily Tao <tao.emily@yahoo.ca>
Earlier we used an empty namespace for data file located
at bundle root. This change now uses "data" as the
namespace for a data file at root.
Fixes: #4022
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
If the following conditions hold for a set of rules returned by the indexer,
it will set EarlyExit: true, and change how the complete virtual doc or
function is evaluated:
- all rule head values are ground
- all rule head values match
This implies that some cases where early exit would be possible will not be
covered:
p = x {
x := true
input.foo == "bar"
}
p = x {
x := true
input.baz == "quz"
}
To indicate that "early exit" is possible, the indexer result message is
amended. Also, the "Exit" trace event will have a message of "early" when
"early exit" actually happens in eval:
$ echo '{"x":"x", "y":"y"}' | opa eval -I -fpretty --explain=full -d r.rego data.r.r
query:1 Enter data.r.r = _
query:1 | Eval data.r.r = _
query:1 | Index data.r.r (matched 2 rules, early exit)
r.rego:11 | Enter data.r.r
r.rego:12 | | Eval input.y = "y"
r.rego:11 | | Exit data.r.r
query:1 | Exit data.r.r = _
query:1 Redo data.r.r = _
query:1 | Redo data.r.r = _
r.rego:11 | Redo data.r.r
r.rego:12 | | Redo input.y = "y"
r.rego:11 | Exit data.r.r early
With `r.rego` as
package r
r {
input.x = "x"
}
r = 2 {
input.z = "z"
}
r {
input.y = "y"
}
This is done in in a way such that early-exit will abort array/set/object
iterations on data:
r {
data.i[_] = "one"
data.j[_] = "four"
}
f(x, y) {
data.i[_] = x
data.j[_] = y
}
Complete rules (r) and functions (f) that iterate over sets, arrays, and
objects from either data (evalTree) or a term that's returned by some
other rule etc (evalTerm).
The CLI and golang packages expose ways to disable 'early-exit':
This is in line with how indexing can be disabled. It's supposed to be
used as a debugging measure, so it's only exposed as a CLI flag to
`opa eval`.
Co-authored-by: Torin Sandall <torinsandall@gmail.com>
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This commit updates the runtime to pass the logger to the plugin
manager and the reporter. In addition, the reporter is updated to pass
the logger into the rest client that it creates. With this change, we
no longer rely on the global logger and the logging configuration is
applied correctly.
To verify that this change is going to fix the problem, I have
searched for references to logrus and the logging package.
Grepping for references to logrus reveals that outside of the logging
package, we only refer to formatters and fields (never the global
logrus logger directly).
Grepping for references to logging.{New, Get, NewStandardLogger}
shows that we only create new loggers if one has not been injected
into the manager or rest client. Since we are passing/injecting the
logger from the runtime, I am fairly confident this will fix the
underlying issue.
Fixes#3958
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This commit adds a new subcommand in opa that inspects
a bundle and summarizes its structure and contents.
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
When `-z` is passed to `opa test`, skipped tests will not affect
the exit code. (When not passed, skipped tests produce the same
exit code as failed tests.)
Fixes#3773.
Signed-off-by: Kirk Patton <kpatton@verizonmedia.com>
A command that previously just ran for a very long time can now be given
a deadline (analogously to `opa test`):
$ opa eval -fpretty --timeout 1s 'numbers.range(1, 100000000)[_] > 10000000'
1 error occurred: eval_cancel_error: numbers.range: timed out before generating all numbers in range
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This adds a new top-level key to the capabilities structure, `allow_net`.
It currently is only used for restricting the typechecker's ability to fetch
remote refs in JSON schemas, but could be used more widely in the future.
It works like this:
- If it's not present, any host can be contacted
- If it's present, the items will be the hosts or IP addresses that may be
contacted; anything not in the list is prohibited.
- As a consequence, If it's present and empty (`[]`), no host can be contacted
Introducing a package-level var to gojsonschema isn't the prettiest solution,
but since we want this in an all-or-nothing way right now anyways, it does
the trick. And it's more ergonomic than adding extra parameters all over the
place.
Fixes#3746.
Also:
* move some profiling-related default params into newEvalCommandParams
* replace some errors.Wrap by fmt.Errorf in loader pkg
* remove some != nil handling where it didn't make a difference when
working on the schema set
* reduces indentation in code examples in `opa eval -h` and `opa check -h`
by replacing tabs by four spaces.
* ast: allow testing with remote refs without networking
It would be nice to ensure that the remote refs feature actually works,
without introducing a network dependency into our tests.
This commit adds the kube 1.14 definitions into ast/testdata, and uses
that from a httptest.Server instance in the unit tests.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
The new helper,
(ResultSet) Allowed() bool
is a shortcut for the common "one value, a bool, indicating allowed-ness"
scenario. To err on the safe side, it'll be false if any bindings are present.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Users of OPA as a library are concerned about big binary blobs in their vendor/
directories. Even more so if they don't use them. This is the case for anyone
using OPA as library, but not using the wasm-backed evaluation feature.
With this change, importers of any packages other than `server` and `cmd`
will have to explicitly opt-in to using wasm evaluation features by having an
underscore import somewhere:
import _ "github.com/open-policy-agent/opa/features/wasm"
Fixes#3545.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This was introduced in #3624: turns out the eval command's code only
passed parsedInput into the rego package, and that only looked at
rawInput when feeding it into the wasm engine.
The approach taken here is to make the rego package more robust: It
should do the right thing if parsedInput is provided for an eval
under the wasm target: the parsing work was already done, let's not
dischard it.
Fixes#3666.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
* Support for minimum TLS version
OPA server now supports min TLS version, TLS versions supported are 1.0, 1.1, 1.2, 1.3.
Since TLS 1.0 and 1.1 are deprecated, default min TLS version for OPA is TLS 1.2 but
if someone wants to restrict OPA to use a specific minimum TLS version, they can
specify it using cmd parameter `--min-tls-version`.
Fixes#3226.
Signed-off-by: Amruta Kale <amruta.kale@styra.com>
This way it will be obvious if users run `opa build` and do not see
their manifest included. Also, update the `opa build` --revision flag
to not get set to the default empty value unconditionally.
Fixes#3480
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
The help text for opa fmt stated that -e could
be used as shorthand. This is not correct.
This commit changed to the correct flag --fail.
Fixes: #3518
Signed-off-by: Andre Håland <andre.haland@gmail.com>
golint is deprecated. The author of the code no longer supports the
codebase. golangci-lint is faster than golint, and is in use by other
opa repositories (e.g. Gatekeeper).
This commit changes tools.go to reference golangci (so it ends up in
vendor) and modifies check-lint to use golangci instead.
Breaking API Changes:
- plugins/rest/rest.go: Fix typo "AllowInsureTLS" -> "AllowInsecureTLS"
- storage/errors.go: Removed unused IndexingNotSupportedErr
Signed-off-by: Will Beason <willbeason@google.com>
* workflow: fix codeql go version
* cmd/fmt_test: use ioutil.Discard
I had proposed that in a PR review... not thinking that it meant
we couldn't run tests using Go 1.15. My assumption that using
go 1.15
in go.mod would protect us from that was wrong, apparently.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Running fmt during CI/CD is a common practice to ensure
files being merged are properly formatted. While this
can be accomplished with shell commands the OPA docker
image doesn't ship with a shell so adding this flag to
return a non zero error code allows the docker image to
be used in CI/CD.
Signed-off-by: Joshua Shanks <jjshanks@gmail.com>
* build: WASM_ENABLED=1 for all platforms, bump go to 1.16.3
Notes:
- If there are other users of the 'build-windows' make target they would
likely be annoyed by the change that's now apt-get'ting packages
- We could build a builder image instead of installing the package every
time.
- ci-go-*: run as root now, so we're able to install the packages for
windows.
- tests: skip tests that depend on not being run as root when root. The
change to ci-go-* makes that necessary; the impact is rather limited
right now. We can reconsider if there are more tests depending on not
being run as root.
- build: add '-buildmode=exe' to GOFLAGS
Primarily for the windows build, but I don't think it should be wrong
for the others either:
https://github.com/golang/go/issues/40795
See https://golang.org/cmd/go/#hdr-Build_modes:
> -buildmode=exe
> Build the listed main packages and everything they import into
> executables. Packages not named main are ignored.
- go: fix version as 1.16.3 (not 1.16)
We'd rather keep this an exact match.
- build: update go module related env vars
With 1.16, https://blog.golang.org/go116-module-changes,
> The go command now builds packages in module-aware mode by default.
Also, since we've added the `go 1.15` directive to go.mod, we can drop
all -mod=vendor flags, https://golang.org/ref/mod#go-mod-file-go,
> At go 1.14 or higher, automatic vendoring may be enabled. If the file
> vendor/modules.txt is present and consistent with go.mod, there is no
> need to explicitly use the -mod=vendor flag.
- build: override docker id/gid in 'image' target, to keep existing
behaviour.
* workflow: use binaries built before, remove workaround
split linux and windows to not wait for the windows build to finish
before starting the npm-opa-wasm tests.
* wasm-sdk: show where to get binaries, don't panic
Fixes#3264.
* Makefile: deprecate old targets, introduce new ones
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This commit combines a bunch of refactoring on annotations to support
future work.
Specifically:
* Annotations are now normal AST nodes/statements. This means that
annotations store locations and also implement String() and
Compare(). Annotations are now correctly compared during module
comparison and annotations are included in the module string
representation (before annotations would be dropped when the module
String() function was called.) Also, the visitor and transformer
functions support annotations now.
* Annotations are no longer hidden behind an interface. Instead, there
is a single annotation struct that we can evolve over
time. It was unclear how the Annotations interface was going to work
in the long-term (e.g., callers would not be able to define their
own annotation types since the parser needs to be aware of them.)
With this change, Annotations are just structs now. We can extend
the struct as needed going forward. Custom data can be stored in a
dedicated field.
* Annotation parsing has been refactored. We now attach annotations to
the statement following the annotation. The parser will reject
METADATA blocks that contain whitespace between the METADATA hint
and the YAML block. Similarly, we no longer support trailing
unindented comments that follow the METADATA block. Users can inject
whitespace after the YAML block if they want to include trailing
comments.
* The opa parse subcommand now enables annotation processing.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Stephan Renatus