A tiny first step to have more tooling correctly report
virtual and base document conflicts, as detailed in #7694.
This PR fixes the `opa check` command to report conflicts
of this type when the `-b`/`--bundle` flag is provided. The
bundle flag is required as without that, `opa check` should
only verify policies and not load data at all.
While I was in the `cmd` directory, I got annoyed with how
many of these commands store the same constants for their
`--format` flag, so I decided to fix that too, even if it
wasn't related to what I originally planned to do. I hope
it's not too distracting.
Signed-off-by: Anders Eknert <anders@styra.com>
Following up on #7566, and now applying the more exciting
modernizations. fmt.Appendf was new to me! But especially
the contains checks are so much better IMHO. I have reviewed
all changes myself and did a few manual changes where it
became obvious that things could be improved a little further.
(the modernize analyzer still has some issues running against
OPA, and I have manually worked around those for the time being)
Signed-off-by: Anders Eknert <anders@styra.com>
Brace yourselves! For there are many touched files here. No changes
in semantics however.
Spent a long time trying out the various optional rules gocritic
provides, and settled for a few of them. There are more I really
like, but that would take many hours to address across the codebase.
Perhaps others find gocritic too pedantic? If so, we can merge the
fixes without enabling the rule.
Signed-off-by: Anders Eknert <anders@styra.com>
By tagging the worst offenders, we can make use of `go test -short` to
avoid them for a quicker dev-test cycle. Compare:
```
make test 200.69s user 209.81s system 170% cpu 4:01.20 total
```
```
make test-short 70.32s user 29.17s system 350% cpu 28.367 total
```
From 4 minutes down to under 30 seconds. The short tests can either
be run with `go test -short ./...` or `make test-short`.
We'll still run the full test suite in CI, naturally.
Also:
- Remove section on benchmarking that linked to a no longer used resource.
Signed-off-by: Anders Eknert <anders@styra.com>
And update code to conform to the rule.
- Replace unnecessary fmt.Sprintf with string concatenation
- Replace fmt.Sprint with more efficient strconv.Itoa
- Replace static fmt.Errorf calls with more efficient errors.New
Thanks @srenatus for pushing me down this rabbit hole!
Signed-off-by: Anders Eknert <anders@styra.com>
All packages, except for `cmd` and `internal`, have been moved into a new `v1` root package.
Old packages are kept for backwards-compatibility reasons. All contained code is replaced with simple type aliases and proxy functions to `v1` implementations.
Old packages default to the Rego v0 syntax, new `v1` packages default to the Rego v1 syntax.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
This isn't needed anymore, so now we don't.
Also enabled the copyloopvar linter in case we
accidentally do this in the future.
Signed-off-by: Anders Eknert <anders@styra.com>
* test: Parallelize package level tests in high-cost packages.
This commit adds `t.Parallel()` calls to the beginning of many tests
across several Go packages in OPA. The slowest packages (taking ~10s or
more) have been instrumented where possible as a proof-of-concept. On a
machine with many cores, the tests now will complete as fast as the
slowest test per package, instead of the sum of all the tests in a
particular package.
* server/server_test: Remove 3x tests from parallel set.
This commit fixes a data race that could occur in the `server` package
tests, because 3x tests were modifying package variables under
`internal/version`. These tests now run sequentially, and are not
included in the parallel test set.
* plugins/bundle/plugin_test: Remove 2x tests from the parallel set.
Two tests in this package modified a package variable directly, and as
such cannot be safely run in parallel with each other or any other tests
in the package.
* topdown/*_test: t.Parallel refactors.
This commit wraps up a large batch of fairly mechanical refactorings to
add t.Parallel() annotations to almost every test under `topdown`. The
tests that could not be safely parallelized now have explicit warning
comments on them describing why they are not safe to run in parallel.
* storage/disk: t.Parallel refactors.
This commit bundles up test parallelization changes for the
`storage/disk` package, dramatically reducing its execution time.
* topdown/net_test: Remove sub-test parallelization.
* rego: t.Parallel refactors.
This commit includes a bundle of t.Parallel refactoring changes for the
`rego` package, including a timer-related bugfix, and a slight change on
a cancellation test to reduce its overall cost during test runs (the
logic is preserved, but the mandatory timeouts are lower now).
* test: Fixes for sporadic test breakages.
---------
Signed-off-by: Philip Conrad <philip@chariot-chaser.net>
To make OPA behave as v0.x post v1.0 release.
If used simultaneously with `--v1-compatible` flag, the `--v0-compatible` flag takes precedence.
Also, future-proofing `cmd` package tests for 1.0.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
When `rego.v1` is in the list of imports directly applied on the `rego.Rego` SDK struct, this import, and it's effects, is applied to the query when parsed.
This change affects the `eval` and `bench` commands when the `--imports` flag is used.
Fixes: #6701
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Adding a global `rego_version` attribute to bundle manifest, to inform OPA runtime about what rego-version (v0/v1) to use to parse/compile contained Rego files.
The rego-version of individual Rego files can be overridden through the `file_rego_versions` manifest attribute.
Implements: #6578
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
In addition to those commands already supported:
* build
* check
* eval
* fmt
* test
support has been added to the following commands:
* `bench`
* `deps`
* `exec`
* `inspect`
* `parse`
* `run` (command `server` and `REPL`)
Fixes: #6520
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Currently the e2e bench mode doesn't support providing an
OPA configuration to enable features like decision logging
that can have an impact on the server overhead. This change
adds a new flag to the bench cmd to specify the OPA configuration.
Fixes: #4899
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
This commit adds a new flag to the opa bench command which
allows users to run benchmarks against a running OPA server.
This mode can be used to evaluate the additional overhead the
server is going to introduce.
Co-authored-by: Anders Eknert anders@eknert.com
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
The new helper,
(ResultSet) Allowed() bool
is a shortcut for the common "one value, a bool, indicating allowed-ness"
scenario. To err on the safe side, it'll be false if any bindings are present.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
golint is deprecated. The author of the code no longer supports the
codebase. golangci-lint is faster than golint, and is in use by other
opa repositories (e.g. Gatekeeper).
This commit changes tools.go to reference golangci (so it ends up in
vendor) and modifies check-lint to use golangci instead.
Breaking API Changes:
- plugins/rest/rest.go: Fix typo "AllowInsureTLS" -> "AllowInsecureTLS"
- storage/errors.go: Removed unused IndexingNotSupportedErr
Signed-off-by: Will Beason <willbeason@google.com>
This change allows users to get an accurate estimate of partial eval
performance on their policies.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This commit replaces the existing PEG generated parser with a parser
implemented by hand. The new parser is more efficient (avoiding old
problems with pathological input cases like {{{{{{{{{}}}}}}}} and
deeply-nested composites in general) and offers better opportunities
for improved error reporting (which has been improved already but
there is still room to grow.)
During the test process of implementing the new parser, we identified
a few issues that were present in the old parser. Those issues are
fixed by this commit.
Fixes#1251Fixes#501Fixes#2198Fixes#2199Fixes#2200Fixes#2201Fixes#2202Fixes#2203
Co-authored-by: Torin Sandall <torinsandall@gmail.com>
Co-authored-by: Patrick East <east.patrick@gmail.com>
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Signed-off-by: Patrick East <east.patrick@gmail.com>
This adds in a new CLI sub command `opa bench` which will load
and evaluate similar to `opa eval` but will perform benchmark testing
of the query.
There is also a new `--bench` option for `opa test` which will
similarly perform benchmarking, except on the unit tests.
Both use the golang testing frameworks benchmark tooling, and the
output format is compliant with the go benchmark standard when using
the newly added `gobench` output format option.
They both support specifying a `--count` to run the benchmark a
number of times and a `--benchmem` option to report memory statistics.
To help enable using the `opa test` command better with the benchmark
option there is now a `--run`/`-r` option that can be provided to
specify a regex for what test cases should be run. The regex supports
anything that is supported by re2:
https://github.com/google/re2/wiki/Syntax
These changes required updating to Go 1.13 to get the ability to
report custom metrics with the benchmark results
https://golang.org/pkg/testing/#B.ReportMetric To get Netlify on board
we needed to add a `.go-version` file to the root of the repo. This is
now the single source of truth for the OPA golang version.
Fixes: #1424
Signed-off-by: Patrick East <east.patrick@gmail.com>
Jacob Hochstetler