Have done this some time in the past, but there was a few
new issues this would highlight now that we're on Go 1.24.
Mostly:
- Use `b.Loop()` in benchmarks
- Use `strings.SplitSeq` where possible
- Remove `omitempty` tag for types that can't be empty
Signed-off-by: Anders Eknert <anders@eknert.com>
Go 1.23 is no longer supported as per Go release policy.
Changes:
- Use Go v1.24.6 as the project SDK requirement
- Apply lint fixes for Go 1.24
- Fix "non-constant format string in call" issues as seen in CI.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
Following up on #7566, and now applying the more exciting
modernizations. fmt.Appendf was new to me! But especially
the contains checks are so much better IMHO. I have reviewed
all changes myself and did a few manual changes where it
became obvious that things could be improved a little further.
(the modernize analyzer still has some issues running against
OPA, and I have manually worked around those for the time being)
Signed-off-by: Anders Eknert <anders@styra.com>
Brace yourselves! For there are many touched files here. No changes
in semantics however.
Spent a long time trying out the various optional rules gocritic
provides, and settled for a few of them. There are more I really
like, but that would take many hours to address across the codebase.
Perhaps others find gocritic too pedantic? If so, we can merge the
fixes without enabling the rule.
Signed-off-by: Anders Eknert <anders@styra.com>
And update code to conform to the rule.
- Replace unnecessary fmt.Sprintf with string concatenation
- Replace fmt.Sprint with more efficient strconv.Itoa
- Replace static fmt.Errorf calls with more efficient errors.New
Thanks @srenatus for pushing me down this rabbit hole!
Signed-off-by: Anders Eknert <anders@styra.com>
Fixing issue in the `opa eval` command where `--v0-compatible` was ignored for PE support modules when
used in combination with the `pretty` output format; producing v1 support modules instead of v0.
Fixing: #7248
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Prioritizing generating v0 Rego with `rego.v1` import when producing support modules for non-`--v1-compatible` optimized builds.
Affects `opa build` when the `-O` flag is used for optimization, and `opa eval` for partial evaluation with the `-p` flag.
Fixes: #6450
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
The number of EVAL/REDO counts in the profile result
are sometimes difficult to understand. This is mainly due to the
fact that the compiler rewrites expressions and assigns the
same location to each generated expression and the profiler
keys the counters by the location. So users have no idea
that multiple expressions may be contributing to the profile
result for a given line in the policy.
This change attempts to provide more clarity to the profile
output by including the number of generated expressions for
each given expression thereby helping to better understand
the result and also how the evaluation works.
Fixes: #2552
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
This commit adds the `prealloc` linter to the list of linters for OPA, and fixes up the miscellaneous locations in the code that the linter found where we could easily preallocate slices.
Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
Add option to inmem.store which allows disabling the round-tripping
through JSON when adding data to the store.
This option is intended for callers who can guarantee the objects they
pass to Write are JSON objects, and have properly ensured the object
will be only be accessed by store once added.
Fixes#4708.
This is continuance of https://github.com/open-policy-agent/opa/pull/4709,
adding these bits:
* storage/inmem: backwards-compat nitpicks, test adaptations
I might have overshot here, but adding variable-length function parameters
is not a backwards-compatible move. Concretely, if you had been using code like
var x func() storage.Store = inmem.New
going from New() to New(...Opts) would break it.
* storage/inmem: use it where possible without roundtrip
* storage/inmem: deal with nil map
It looks like this is something the roundtrip had guarded us from.
Now, we'll explicitly check this.
This came up when running the bundle tests with roundtripping disabled.
* loader: add StoreWithOpts convenience method
Co-authored-by: Will Beason <willbeason@google.com>
Co-authored-by: Philip Conrad <conradp@chariot-chaser.net>
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This commit adds support for named argument declarations for built-in
functions as well as additional metadata/annotations on built-in
functions (e.g., descriptions, categories, etc.) This commit allows us
to generate a data file (builtin_metadata.json) that other tools can
consume to improve the Rego authoring experience.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Co-authored-by: Torin Sandall <torinsandall@gmail.com>
* format: allow ignoreing source locations
* cmd/eval: format disregarding source locations for partial result
Before, we'd see this output:
```
$ opa eval -p -fsource 'time.clock(input.x)==time.clock(input.y)'
# Query 1
time.clock(time.clock(input.x), input.y)
```
Now, we get the proper answer: `time.clock(input.y, time.clock(input.x))`.
Note that it's a _display_ issue; the JSON output of PE has not been affected.
Fixes#4609.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This change also modifies the tests to use reflect.DeepEqual to
compare JSON output instead of raw string comparison. This seems
preferable as it insulates the tests from changes in whitespace (which
are not meaningful when dealing with JSON.)
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
* types: Sort any elements during construction
This changes updates the implementation of the any type to sort
elements during construction. This way the Compare() function does not
have to sort elements before recursing (which can result in data races
if global type instances from the built-in function declarations or
elsewhere are compared.)
Fixes#3793
* capabilities.json: changed ordering
* internal/presentation: fix json error output
Co-authored-by: Torin Sandall <torinsandall@gmail.com>
Co-authored-by: Stephan Renatus <stephan.renatus@gmail.com>
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Before, errors displayed through the presentation package's prettyError()
method would never show their details. The expectation in the code was
that any error that has details would include them in its own Error()
string method.
When error types return their details using an interface, such as
type ErrorDetail interface {
Lines() []string
}
and their Error() method only return a short error message, then that
detail would never make it to the user in any of the non-JSON output
variants.
Before:
$ opa eval -t wasm '1+1'
{
"errors": [
{
"message": "engine not found",
"details": "WebAssembly runtime not supported in this build.\n----------------------------------------------------------------------------------\nPlease download an OPA binary with Wasm enabled from\nhttps://www.openpolicyagent.org/docs/latest/#running-opa\nor build it yourself (with Wasm enabled).\n----------------------------------------------------------------------------------\n"
}
]
}
$ opa -fpretty -t wasm '1+1'
1 error occurred: engine not found
Now:
$ opa eval -t wasm '1+1'
{
"errors": [
{
"message": "engine not found",
"details": "WebAssembly runtime not supported in this build.\n----------------------------------------------------------------------------------\nPlease download an OPA binary with Wasm enabled from\nhttps://www.openpolicyagent.org/docs/latest/#running-opa\nor build it yourself (with Wasm enabled).\n----------------------------------------------------------------------------------"
}
]
}
$ opa eval -fpretty -t wasm '1+1'
1 error occurred: engine not found
WebAssembly runtime not supported in this build.
----------------------------------------------------------------------------------
Please download an OPA binary with Wasm enabled from
https://www.openpolicyagent.org/docs/latest/#running-opa
or build it yourself (with Wasm enabled).
----------------------------------------------------------------------------------
This also surfaces the error details in REPL sessions:
$ opa run
OPA 0.32.0-dev (commit 3da95f9c-dirty, built at 2021-08-16T11:24:46Z)
Run 'help' to see a list of commands and check for updates.
> target wasm
> true
1 error occurred: engine not found
WebAssembly runtime not supported in this build.
----------------------------------------------------------------------------------
Please download an OPA binary with Wasm enabled from
https://www.openpolicyagent.org/docs/latest/#running-opa
or build it yourself (with Wasm enabled).
----------------------------------------------------------------------------------
>
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Users of OPA as a library are concerned about big binary blobs in their vendor/
directories. Even more so if they don't use them. This is the case for anyone
using OPA as library, but not using the wasm-backed evaluation feature.
With this change, importers of any packages other than `server` and `cmd`
will have to explicitly opt-in to using wasm evaluation features by having an
underscore import somewhere:
import _ "github.com/open-policy-agent/opa/features/wasm"
Fixes#3545.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
golint is deprecated. The author of the code no longer supports the
codebase. golangci-lint is faster than golint, and is in use by other
opa repositories (e.g. Gatekeeper).
This commit changes tools.go to reference golangci (so it ends up in
vendor) and modifies check-lint to use golangci instead.
Breaking API Changes:
- plugins/rest/rest.go: Fix typo "AllowInsureTLS" -> "AllowInsecureTLS"
- storage/errors.go: Removed unused IndexingNotSupportedErr
Signed-off-by: Will Beason <willbeason@google.com>
This is a very simple addition but it's proven extremely useful to us since
it means you can now use `opa` easily in bash scripts, e.g.:
```bash
ACCOUNT_ID="$(opa eval -d accounts.rego --format raw "data.accounts.account")"
```
This means `accounts.rego` can be used by policies as well as e.g. deploy
scripts.
The naming and functionality is inspired by the `-r` flag from `jq`. In fact,
`jq` can be used to replicate this behaviour, but it is a bit nicer to not rely
on that system dependency.
When multiple queries and expressions are given, they are printed in a simple
table format using single newlines and spaces since that is consistent and plays
nice with `bash`.
Signed-off-by: Jasper Van der Jeugt <jasper@fugue.co>
Signed-off-by: timakin <timaki.st@gmail.com>
through if at least one of expressions has a non-boolean value
Signed-off-by: timakin <timaki.st@gmail.com>
defined boolRenderable func which checks boolean Values and Bindings contents
Signed-off-by: timakin <timaki.st@gmail.com>
added a test for boolean flags pretty printing
Signed-off-by: timakin <timaki.st@gmail.com>
add a newline at end of file
Signed-off-by: timakin <timaki.st@gmail.com>
adda newline at end of file
Signed-off-by: timakin <timaki.st@gmail.com>
use len(rs[0].Bindings) to simplify functions
Signed-off-by: timakin <timaki.st@gmail.com>
simplified selectVarValue
Signed-off-by: timakin <timaki.st@gmail.com>
delete unused the existence flag of selectVarValue result
Signed-off-by: timakin <timaki.st@gmail.com>
delete an empty line
Signed-off-by: timakin <timaki.st@gmail.com>
delete an empty line
Signed-off-by: timakin <timaki.st@gmail.com>
This commit replaces the existing PEG generated parser with a parser
implemented by hand. The new parser is more efficient (avoiding old
problems with pathological input cases like {{{{{{{{{}}}}}}}} and
deeply-nested composites in general) and offers better opportunities
for improved error reporting (which has been improved already but
there is still room to grow.)
During the test process of implementing the new parser, we identified
a few issues that were present in the old parser. Those issues are
fixed by this commit.
Fixes#1251Fixes#501Fixes#2198Fixes#2199Fixes#2200Fixes#2201Fixes#2202Fixes#2203
Co-authored-by: Torin Sandall <torinsandall@gmail.com>
Co-authored-by: Patrick East <east.patrick@gmail.com>
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Signed-off-by: Patrick East <east.patrick@gmail.com>
This corrects the missing time in rego_module_parse timers as we now
have metrics collecting info as we parse *.rego files from file
loaders and from bundles as they are unpacked.
It also adds in a timer for the data files that are loaded through
similar mechanisms.
Signed-off-by: Patrick East <east.patrick@gmail.com>
Previously if the errors passed into the presentation Output were not
structured w/ JSON tags for marshaling the error would be an empty
string.
This changes to wrap the errors with a struct in cases where they
would otherwise not be formatted. We do this by forcing every error
into a structure and translating known error types into it.
Fixes: #1726Fixes: #1724
Signed-off-by: Patrick East <east.patrick@gmail.com>
Previously coverage was only available via the test runner. This
allows callers to run ad-hoc queries and see what parts of the policy
are evaluated.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
For the time being, enabling profiling will disable tracing and vice versa. Once we add support for multiple tracers, this behavior can be changed.
Fixes#838
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Signed-off-by: repenno <rapenno@gmail.com>
Previously, the format package would return an error if any of the AST
nodes under the input were missing a location value. When the format
package was first implemented, the main use case was formatting policies
that people had written manually--which means they are provided to OPA
as files/raw strings. As a result, it made sense to treat a missing
location as an error condition because it simplifies the formatting
implementation.
However, when policies are generated (e.g., by partial evaluation) the
AST nodes do not typically carry locations. As a result, these AST nodes
cannot be formatted nicely.
These changes modify the format package to tolerate nil location values.
If a nil location value is encountered, the format package will set the
location value on the AST node to a default location, currently row 1
column 1 with text from the AST node's string representation.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously, if an empty ast.Body was passed to the formatting package,
it would trigger a panic because the location getter would try to index
into an empty slice.
These changes make the location getter tolerate empty bodies and the
format package tolerate nil locations on empty bodies. The changes also
improve simplify the error message when nil locations are found.
Fixes#909
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
These changes add a new subcommand that analyzes policies and prints
base and virtual document dependencies.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
These changes update the eval subcommand to support partial evaluation.
As part of these changes, the pretty formatting of partial evaluation
results has been refactored and moved into the presentation package. The
new version uses the tablewriter like other output values.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
The presentation implementation was trying to print a table with no
expression values or variable bindings. With these changes, the
presentation package will output 'true' if there are no vars and all of
the exprs are of type 'bool'.
Fixes#850
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously, output was being printed in the presentation package, REPL,
and eval subcommand. Thes changes refactor the presentation package so
that it can handle all of the output printing required by the REPL and
the eval subcommand.
These changes affect the 'json' output format in the REPL. Previously,
the JSON output format would display either the expression value or
bindings. With these changes, the 'json' output format in the REPL is
the same as the one in the eval subcommand.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Stephan Renatus