Go 1.23 is no longer supported as per Go release policy.
Changes:
- Use Go v1.24.6 as the project SDK requirement
- Apply lint fixes for Go 1.24
- Fix "non-constant format string in call" issues as seen in CI.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
All packages, except for `cmd` and `internal`, have been moved into a new `v1` root package.
Old packages are kept for backwards-compatibility reasons. All contained code is replaced with simple type aliases and proxy functions to `v1` implementations.
Old packages default to the Rego v0 syntax, new `v1` packages default to the Rego v1 syntax.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Also making some updates to the repl implementation to properly deal with v1 as the default rego-version.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
In addition to those commands already supported:
* build
* check
* eval
* fmt
* test
support has been added to the following commands:
* `bench`
* `deps`
* `exec`
* `inspect`
* `parse`
* `run` (command `server` and `REPL`)
Fixes: #6520
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
This commit changes the behavior of the function
`newCommand` in `repl.go`. The input of this
function is a string containing both a command
to be run and its arguments. Prior to this commit,
the function converted the entire input to lower
case, without any distinction between the command
and its arguments. This lead to the bug exposed
in issue #5229.
Both the lowercase command and all lower case
arguments are put as fields in the `command`
struct returned by the function.
After this commit, the only part of the string that
is converted to lower case is the command that is
being executed, while the provided arguments are
evaluated as provided to the function. The struct
returned now contains the lower
case command and the arguments as provided to the
function.
Fixes: #5229
Signed-off-by: Gianluca Oldani <oldanigianluca@gmail.com>
This commit replaces `os.MkdirTemp` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.
Prior to this commit, temporary directory created using `os.MkdirTemp`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
defer func() {
if err := os.RemoveAll(dir); err != nil {
t.Fatal(err)
}
}
is also tedious, but `t.TempDir` handles this for us nicely.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
This would be useful for us for two immediate use cases:
1. Show how and why rules failed in more detail in verbose tooling,
we can show the unification happening step by step.
2. We can trace which parts of the input document were used, if we
add `Location` info those terms.
However, I think it's generally useful for debugging tools.
This increases verbosity in the explain logs, so we decided to add a new explain
mode `debug` in addition to the existing `full`, `notes`, `fails`, `off` modes.
This can be set using the `--explain=debug` flag on the CLI, or by using `trace
debug` in the REPL.
Signed-off-by: Jasper Van der Jeugt <m@jaspervdj.be>
Add option to inmem.store which allows disabling the round-tripping
through JSON when adding data to the store.
This option is intended for callers who can guarantee the objects they
pass to Write are JSON objects, and have properly ensured the object
will be only be accessed by store once added.
Fixes#4708.
This is continuance of https://github.com/open-policy-agent/opa/pull/4709,
adding these bits:
* storage/inmem: backwards-compat nitpicks, test adaptations
I might have overshot here, but adding variable-length function parameters
is not a backwards-compatible move. Concretely, if you had been using code like
var x func() storage.Store = inmem.New
going from New() to New(...Opts) would break it.
* storage/inmem: use it where possible without roundtrip
* storage/inmem: deal with nil map
It looks like this is something the roundtrip had guarded us from.
Now, we'll explicitly check this.
This came up when running the bundle tests with roundtripping disabled.
* loader: add StoreWithOpts convenience method
Co-authored-by: Will Beason <willbeason@google.com>
Co-authored-by: Philip Conrad <conradp@chariot-chaser.net>
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
* ast/compile: check arity in undefined function stage
Before, the "undefined function" check stage in the compiler (and query
compiler) only asserted that the function was known.
Now, we'll also check that the number of arguments _could be_ valid. If
it really is valid will be determined by the type checker at a later
stage.
However, asserting the arity early allows us to give more on-the-spot
error messages.
Fixes#4054.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
If the following conditions hold for a set of rules returned by the indexer,
it will set EarlyExit: true, and change how the complete virtual doc or
function is evaluated:
- all rule head values are ground
- all rule head values match
This implies that some cases where early exit would be possible will not be
covered:
p = x {
x := true
input.foo == "bar"
}
p = x {
x := true
input.baz == "quz"
}
To indicate that "early exit" is possible, the indexer result message is
amended. Also, the "Exit" trace event will have a message of "early" when
"early exit" actually happens in eval:
$ echo '{"x":"x", "y":"y"}' | opa eval -I -fpretty --explain=full -d r.rego data.r.r
query:1 Enter data.r.r = _
query:1 | Eval data.r.r = _
query:1 | Index data.r.r (matched 2 rules, early exit)
r.rego:11 | Enter data.r.r
r.rego:12 | | Eval input.y = "y"
r.rego:11 | | Exit data.r.r
query:1 | Exit data.r.r = _
query:1 Redo data.r.r = _
query:1 | Redo data.r.r = _
r.rego:11 | Redo data.r.r
r.rego:12 | | Redo input.y = "y"
r.rego:11 | Exit data.r.r early
With `r.rego` as
package r
r {
input.x = "x"
}
r = 2 {
input.z = "z"
}
r {
input.y = "y"
}
This is done in in a way such that early-exit will abort array/set/object
iterations on data:
r {
data.i[_] = "one"
data.j[_] = "four"
}
f(x, y) {
data.i[_] = x
data.j[_] = y
}
Complete rules (r) and functions (f) that iterate over sets, arrays, and
objects from either data (evalTree) or a term that's returned by some
other rule etc (evalTerm).
The CLI and golang packages expose ways to disable 'early-exit':
This is in line with how indexing can be disabled. It's supposed to be
used as a debugging measure, so it's only exposed as a CLI flag to
`opa eval`.
Co-authored-by: Torin Sandall <torinsandall@gmail.com>
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
The future.keywords import can be accessed via the repl in the same
way any other import can, and is handled accordingly:
> show
no rules defined
> import future.keywords.in
> show
package repl
import future.keywords.in
> 1 in [true]
false
> r { input in data.foo }
Rule 'r' defined in package repl. Type 'show' to see rules.
> show
package repl
import future.keywords.in
r {
input in data.foo
}
>
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
golint is deprecated. The author of the code no longer supports the
codebase. golangci-lint is faster than golint, and is in use by other
opa repositories (e.g. Gatekeeper).
This commit changes tools.go to reference golangci (so it ends up in
vendor) and modifies check-lint to use golangci instead.
Breaking API Changes:
- plugins/rest/rest.go: Fix typo "AllowInsureTLS" -> "AllowInsecureTLS"
- storage/errors.go: Removed unused IndexingNotSupportedErr
Signed-off-by: Will Beason <willbeason@google.com>
This commit adds a target flag to the
bench, eval, test and run (repl) commands
which allows users to exercise the wasm
rumtime.
Fixes#2878
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
This commit updates the OPA `run` and `version` commands to report the
version of the running OPA instance to an external service.
In case of the `opa run` command, this feature is ON by-default and
can be disabled using the --skip-version-check flag. In the server mode,
reports are sent periodically while in repl mode only once at start-up.
In case of the opa version command, this feature can be enabled by
specifying the --check or -c flag.
Reports are sent to the configurable external service
on a best-effort basis.
Fixes#1253
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
Signed-off-by: timakin <timaki.st@gmail.com>
through if at least one of expressions has a non-boolean value
Signed-off-by: timakin <timaki.st@gmail.com>
defined boolRenderable func which checks boolean Values and Bindings contents
Signed-off-by: timakin <timaki.st@gmail.com>
added a test for boolean flags pretty printing
Signed-off-by: timakin <timaki.st@gmail.com>
add a newline at end of file
Signed-off-by: timakin <timaki.st@gmail.com>
adda newline at end of file
Signed-off-by: timakin <timaki.st@gmail.com>
use len(rs[0].Bindings) to simplify functions
Signed-off-by: timakin <timaki.st@gmail.com>
simplified selectVarValue
Signed-off-by: timakin <timaki.st@gmail.com>
delete unused the existence flag of selectVarValue result
Signed-off-by: timakin <timaki.st@gmail.com>
delete an empty line
Signed-off-by: timakin <timaki.st@gmail.com>
delete an empty line
Signed-off-by: timakin <timaki.st@gmail.com>
Previously the note events would not have a location on them which
mean they were difficult to track down (you would have to grep for the
message and hope it shows up.) With this change we just include the
location on notes like all other events.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously we had a hard set width for location info in pretty trace
outputs. This changes to use a dynamic width up to a reasonable max
and then starts to shorten paths as possible by swapping in `...` for
the longest common substring of all paths.
To get the longest substring this brings in a couple files from
https://github.com/vmarkovtsev/go-lcss which implements an efficient
algorithm for it (rather than us implementing something fancy from
scratch). It's pretty isolated and is unlikely to need any updates
over time so the maintenance should be low.
Fixes: #2143
Signed-off-by: Patrick East <east.patrick@gmail.com>
This command is useful for removing packages from
the current REPL session without having to exit it
Fixes#2140
Signed-off-by: Frederic <frederic.vanreet@icloud.com>
The REPL's internal state was getting corrupted if an invalid unknown
term was given. For example `unknown x-1` would result in the unknown
set being allocated but it would contain an illegal nil element.
This fix just updates the REPL to avoid corrupting the internal state
if any of the unknown arguments are invalid.
Fixes#1670
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
While adding support for the some keyword in the REPL and fixing #1104
there was a regression where statements like `input = 1` would always
be interpreted as rules. We made a decision a long time ago that the
first time an expression like `input = 1` was encountered that a rule
would be declared but that subsequent similar expressions (e.g., input
= 1 or input = {"foo":"bar"} or ...) would perform a comparison. The
regression broke this for cases where the left hand side was a
reference to a global document (i.e., input or data). This commit just
fixes the regression by updating the global check to account for refs.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This avoids duplicating any logic that may be required to construct
rules from := expressions. Currently the only extra bit of logic is to
set the assignment flag on the rule head. This change lets us
determine whether the rules are unset in the REPL in a more
declarative manner (i.e., if it's an assignment rule then it will
unset in all cases) and ensure that the assignment operator is not
lost in the show command output.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Also, refactor how the REPL prints debug state a bit to reduce
boilerplate for each possible explanation mode.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This commit renames the 'var' keyword to 'some'. 'some' is more
descriptive than 'var' and will better complement an 'every' or
'forall' keyword representing for universal quantifiers.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously the build version was recorded in the version package and
then different components would report it in an ad-hoc manner, e.g.,
the REPL has a module that generates a virtual doc with the version
info in it, the server was using templating to do the same, etc.
These changes remove the special code from the REPL and server
implementations to report the version. Instead the runtime writes the
version into /system/version at boot.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This was causing a panic because the AST helper to convert the
expression into a rule was (rightly) assuming the operands would be
non-nil. The REPL should just ignore the expression if it's not
well-formed.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously the topdown evaluator only supported a single tracer. As a
result it was not easy to use multiple trace-based features (e.g.,
tracing and profiling) in conjunction.
These changes modify the evaluator to support multiple tracers. Instead
of adding a new interface to register N tracers, these changes just
overload the existing WithTracer function to add the passed tracer.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
For the time being, enabling profiling will disable tracing and vice versa. Once we add support for multiple tracers, this behavior can be changed.
Fixes#838
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Signed-off-by: repenno <rapenno@gmail.com>
Previously users did not have a way to tell if features like tracing were enabled. These changes add a simple REPL command to echo the state of the REPL settings.
Fixes#750
Signed-off-by: repenno <rapenno@gmail.com>
Previously the REPL would just silently define rules. This is a bit
confusing for new users--because they come to assume that x = y should
define a rule but then become confused when x = y acts as a query.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
These changes update the eval subcommand to support partial evaluation.
As part of these changes, the pretty formatting of partial evaluation
results has been refactored and moved into the presentation package. The
new version uses the tablewriter like other output values.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously, output was being printed in the presentation package, REPL,
and eval subcommand. Thes changes refactor the presentation package so
that it can handle all of the output printing required by the REPL and
the eval subcommand.
These changes affect the 'json' output format in the REPL. Previously,
the JSON output format would display either the expression value or
bindings. With these changes, the 'json' output format in the REPL is
the same as the one in the eval subcommand.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously, the REPL command to declare unknowns was 'partial' which is
somewhat confusing. These changes just rename the command to 'unknown'.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
The initial assignment support in the REPL was using the expr operand
instead of the rule name for the unset operation. As a result,
assignments to input/data would panic because the expr operand was a ref
and not a var.
With these changes, the REPL can now print expression values more
reliably. E.g., simple expressions like 3+5 just do the right thing.
Previously the REPL called topdown directly and reimplemented some of
the logic to format result sets. This was a source of issues because it
was possible for the rego package and the REPL to return different
answers. With these changes, the REPL and rego package results are
equivalent.
A few changes were required. Specifically:
* Query Compiler. Updated to accept user supplied stages. This way users
can perform their own rewriting. This is used by the rego package to
provide the query+functional semantics we want. In the future, this API
could be used to register custom optimization passes to the compiler.
* Compiler. Expose GetArity helper. This allows users to quickly lookup
the arity of a function referred to by a ref. The rego package needs
this to decide whether to capture call outputs.
* Rego package. Expose new args to set parse package, imports, etc. This
is used by the REPL which maintains state to control the currently
active module.
Refactor how output vars are computed for call expressions. Previously
the number of rule args were not used to determine which args were
considered outputs. Instead, it was assumed the last arg in the call was
an output. This meant that if an arg in the output position was omitted,
an input arg would be incorrectly marked safe.
With these changes, the compiler looks up the number of args (arity) of
the rule when checking whether an arg is an output.
Previously OPA only tracked query performance a high level (e.g., parse,
compile, eval latencies.) In some cases, it's necessary to instrument
lower level evaluation operations to understand performance. These
changes update the eval implementation to support instrumentation:
* Eval has been instrumented to record time taken for various core
operations like term plugging, reading from the store, rule lookup,
cache hits, etc.
* Rego package has been updated to support a simple rego.Instrument
operation that enables query instrumentation.
* REPL and server have been updated to expose simple interfaces to turn
on instrumentation.
* Diagnostic policy config "all" will enable instrumentation.
Instrumentation can be expensive (because it requires timing frequently
executed operations) so it should be treated as a debugging tool and not
enabled all of the time.
Ville Vesilehto