Have done this some time in the past, but there was a few
new issues this would highlight now that we're on Go 1.24.
Mostly:
- Use `b.Loop()` in benchmarks
- Use `strings.SplitSeq` where possible
- Remove `omitempty` tag for types that can't be empty
Signed-off-by: Anders Eknert <anders@eknert.com>
This change allows users that build their own executable or "spin" of
OPA to give it a name, and have it reference itself properly in help
texts.
It's a vanity thing, but I think some people would appreciate it, hat
tip to the international association of pedants.
Signed-off-by: Stephan Renatus <stephan@styra.com>
Co-authored-by: kevinstyra <83973046+kevinstyra@users.noreply.github.com>
`os.Exit` immediately exits the program and doesn't run defer functions.
This can be problematic as any command.OnFinalize routines and any logic
after the command.Execute won't be run.
Also suppress all RunE cobra error and usage messages. These would be
printed twice otherwise.
Signed-off-by: Stephan Renatus <stephan@styra.com>
Co-authored-by: Kevin St. Pierre <kevin@styra.com>
A tiny first step to have more tooling correctly report
virtual and base document conflicts, as detailed in #7694.
This PR fixes the `opa check` command to report conflicts
of this type when the `-b`/`--bundle` flag is provided. The
bundle flag is required as without that, `opa check` should
only verify policies and not load data at all.
While I was in the `cmd` directory, I got annoyed with how
many of these commands store the same constants for their
`--format` flag, so I decided to fix that too, even if it
wasn't related to what I originally planned to do. I hope
it's not too distracting.
Signed-off-by: Anders Eknert <anders@styra.com>
It's been irritating me for long how `opa bench` has such a high baseline
metric for even the most trivial queries, as in order to know the cost of
"your" Rego you'll need to first subtract the number OPA adds for just
getting eval set up. This improves this somewhat by not initiating some
caches until they're needed. We don't need to cache comprehensions to eval
the value '1', or a functionMockStack, and so on. In fact, we may never
need one. The gain here is miniscule for real policy evaluation, but helps
some with making `opa bench` approach a more reasonable baseline.
We *can* have 10 allocs more removed if we initialize and reuse a base
cache and a virtual cache across all runs. This works as the query is
the same for all runs. However, since those are normally initialized
per "run" (query), perhaps that's going too far?
```
opa bench 1
```
**Before**
```
+-------------------------------------------+------------+
| samples | 398083 |
| ns/op | 2978 |
| B/op | 3200 |
| allocs/op | 49 |
+-------------------------------------------+------------+
```
**After**
```
+-------------------------------------------+------------+
| samples | 432841 |
| ns/op | 2825 |
| B/op | 2968 |
| allocs/op | 40 |
+-------------------------------------------+------------+
```
This change also fixes a panic which happened when the `--metrics` flag
was set to `false`.
Signed-off-by: Anders Eknert <anders@styra.com>
And update code to conform to the rule.
- Replace unnecessary fmt.Sprintf with string concatenation
- Replace fmt.Sprint with more efficient strconv.Itoa
- Replace static fmt.Errorf calls with more efficient errors.New
Thanks @srenatus for pushing me down this rabbit hole!
Signed-off-by: Anders Eknert <anders@styra.com>
And use them to reduce imperative boilerplate throughout
the codebase.
Additionally, replace use of sort.Slice with slices.SortFunc
which is more efficient since it is generic and as such avoids
allocations related to `interface{}` casts.
Also a few performance-related minor fixes, but not the main
theme of this PR.
```
BenchmarkRegalLintingItself-10 before / after
1832684458 ns/op 3453470360 B/op 66125422 allocs/op
1826601250 ns/op 3449619024 B/op 65999164 allocs/op
````
Signed-off-by: Anders Eknert <anders@styra.com>
A new optimized read mode has been added to the default in-memory store, where data written to the store is eagerly converted to AST values (the data format used during evaluation). This pre-converted data is faster to read, and won’t cause memory spikes during load; but comes with slower data writes (affects startup and bundle load/update time) and a larger lowest overall memory footprint for OPA. Can be enabled for `opa run`, `opa eval`, and `opa bench` by setting the `—optimize-store-for-read-speed`. See http://localhost:8888/docs/edge/policy-performance/#storage-optimization.
Implements: #4147
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Co-authored-by: Ashutosh Narkar <anarkar4387@gmail.com>
To make OPA behave as v0.x post v1.0 release.
If used simultaneously with `--v1-compatible` flag, the `--v0-compatible` flag takes precedence.
Also, future-proofing `cmd` package tests for 1.0.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
In addition to those commands already supported:
* build
* check
* eval
* fmt
* test
support has been added to the following commands:
* `bench`
* `deps`
* `exec`
* `inspect`
* `parse`
* `run` (command `server` and `REPL`)
Fixes: #6520
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Currently the e2e bench mode doesn't support providing an
OPA configuration to enable features like decision logging
that can have an impact on the server overhead. This change
adds a new flag to the bench cmd to specify the OPA configuration.
Fixes: #4899
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
We're now using port 0 to have the OS select an open port for us,
and wait in a loop (with exponential backoff) until the runtime returns
an address, so we can go on.
Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
This commit adds a new flag to the opa bench command which
allows users to run benchmarks against a running OPA server.
This mode can be used to evaluate the additional overhead the
server is going to introduce.
Co-authored-by: Anders Eknert anders@eknert.com
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
This adds a new top-level key to the capabilities structure, `allow_net`.
It currently is only used for restricting the typechecker's ability to fetch
remote refs in JSON schemas, but could be used more widely in the future.
It works like this:
- If it's not present, any host can be contacted
- If it's present, the items will be the hosts or IP addresses that may be
contacted; anything not in the list is prohibited.
- As a consequence, If it's present and empty (`[]`), no host can be contacted
Introducing a package-level var to gojsonschema isn't the prettiest solution,
but since we want this in an all-or-nothing way right now anyways, it does
the trick. And it's more ergonomic than adding extra parameters all over the
place.
Fixes#3746.
Also:
* move some profiling-related default params into newEvalCommandParams
* replace some errors.Wrap by fmt.Errorf in loader pkg
* remove some != nil handling where it didn't make a difference when
working on the schema set
* reduces indentation in code examples in `opa eval -h` and `opa check -h`
by replacing tabs by four spaces.
* ast: allow testing with remote refs without networking
It would be nice to ensure that the remote refs feature actually works,
without introducing a network dependency into our tests.
This commit adds the kube 1.14 definitions into ast/testdata, and uses
that from a httptest.Server instance in the unit tests.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
golint is deprecated. The author of the code no longer supports the
codebase. golangci-lint is faster than golint, and is in use by other
opa repositories (e.g. Gatekeeper).
This commit changes tools.go to reference golangci (so it ends up in
vendor) and modifies check-lint to use golangci instead.
Breaking API Changes:
- plugins/rest/rest.go: Fix typo "AllowInsureTLS" -> "AllowInsecureTLS"
- storage/errors.go: Removed unused IndexingNotSupportedErr
Signed-off-by: Will Beason <willbeason@google.com>
This commit adds a target flag to the
bench, eval, test and run (repl) commands
which allows users to exercise the wasm
rumtime.
Fixes#2878
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
New support to input a JSON schema file via "opa eval --schema" that
helps improve static type checking and returns precise error reports as
you develop Rego code.
Also, uses modified version of https://github.com/xeipuuv/gojsonschema
from internal/ to help set Rego types, using gojsonschema's Compiler
results. See README docs for more details on the feature.
Co-authored-by: Mandana Vaziri <mvaziri@us.ibm.com>
Co-authored-by: Ansu Varghese <avarghese@us.ibm.com>
Signed-off-by: Ansu Varghese <avarghese@us.ibm.com>
This change allows users to get an accurate estimate of partial eval
performance on their policies.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This adds in a new CLI sub command `opa bench` which will load
and evaluate similar to `opa eval` but will perform benchmark testing
of the query.
There is also a new `--bench` option for `opa test` which will
similarly perform benchmarking, except on the unit tests.
Both use the golang testing frameworks benchmark tooling, and the
output format is compliant with the go benchmark standard when using
the newly added `gobench` output format option.
They both support specifying a `--count` to run the benchmark a
number of times and a `--benchmem` option to report memory statistics.
To help enable using the `opa test` command better with the benchmark
option there is now a `--run`/`-r` option that can be provided to
specify a regex for what test cases should be run. The regex supports
anything that is supported by re2:
https://github.com/google/re2/wiki/Syntax
These changes required updating to Go 1.13 to get the ability to
report custom metrics with the benchmark results
https://golang.org/pkg/testing/#B.ReportMetric To get Netlify on board
we needed to add a `.go-version` file to the root of the repo. This is
now the single source of truth for the OPA golang version.
Fixes: #1424
Signed-off-by: Patrick East <east.patrick@gmail.com>
Anders Eknert