Disabled by default. To enable, `not` future keyword must be present in capabilities and imported into Rego module.
Implements: #8391
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Adding string interpolation support to the Rego language.
An interpolated string is composed of a template-string that can contain zero or more template-expressions that interpolates values into the string generated at eval-time.
Requires the `template_strings` capability feature and `internal.template_string` built-in function.
Implements: #4733
Parsing is generally fast, so this mainly improves performance
of creating big bundles with many Rego files in them. For Regal's
embedded bundle, loading it from memory would previously take 16
ms on my laptop, and now it takes 9 ms. There are other things
in this process that could be concurrent too, like JSON unmarshalling
of multiple data files. But starting with parsing modules.
This PR adds `errgroup` as a direct dependency (previously indirect)
as it is a nicer way to work with wait groups, and one that can be
useful elsewhere in the codebase (like in the compiler).
Also, and as usual, went off on a bit of a tangent refactoring code
related to the bundle build process, and made sure to use some common
helpers in code where available.
Signed-off-by: Anders Eknert <anders.eknert@apple.com>
Due the way that wasmtime-go does its versioning, it seems to fly under
the radar of dependabot: that will never propose major version bumps, I
suppose.
Signed-off-by: Stephan Renatus <stephan@styra.com>
Go 1.23 is no longer supported as per Go release policy.
Changes:
- Use Go v1.24.6 as the project SDK requirement
- Apply lint fixes for Go 1.24
- Fix "non-constant format string in call" issues as seen in CI.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
Closing the circle here, or something.
Not a lot of Rego used in OPA yet, but some in examples and tests. The little
there is should be linted though, and it'd be good if any new addition of policies got
linted by default. But more than anything, the "ignore configuration" provided here
avoids having developers seeing thousands of issues reported by Regal when they
open the OPA project in VS Code or their editor of choice.
Someone might want to look into un-ignoring the doc directory at some point, as it's
probably a good idea to have the docs follow best practices.
Signed-off-by: Anders Eknert <anders@styra.com>
Following up on #7566, and now applying the more exciting
modernizations. fmt.Appendf was new to me! But especially
the contains checks are so much better IMHO. I have reviewed
all changes myself and did a few manual changes where it
became obvious that things could be improved a little further.
(the modernize analyzer still has some issues running against
OPA, and I have manually worked around those for the time being)
Signed-off-by: Anders Eknert <anders@styra.com>
This change still follows the approach thought up in #6996, but now does
it more consistently: the extra args accumulated through (multiple)
with-replacements using variables are now put into the funcstackj, and
consistently affect the planning of functions in "higher" gens.
Fixes#5311.
Signed-off-by: Stephan Renatus <stephan@styra.com>
And update code to conform to the rule.
- Replace unnecessary fmt.Sprintf with string concatenation
- Replace fmt.Sprint with more efficient strconv.Itoa
- Replace static fmt.Errorf calls with more efficient errors.New
Thanks @srenatus for pushing me down this rabbit hole!
Signed-off-by: Anders Eknert <anders@styra.com>
All packages, except for `cmd` and `internal`, have been moved into a new `v1` root package.
Old packages are kept for backwards-compatibility reasons. All contained code is replaced with simple type aliases and proxy functions to `v1` implementations.
Old packages default to the Rego v0 syntax, new `v1` packages default to the Rego v1 syntax.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
The previous version has been failing without any good reason for me,
so let's try this.
About the version pick: It's not the latest version (v1.62.0 at the
moment), because that would introduce a new revive rule,
redeclares-builtin-id, and that flags every variable called `min` or
`max` in the code base. I had started addressing these, but they were
just too many.
The new issues related to this version are mostly that it complains
whenever it finds a non-static string that makes its way into a printf-
like function. However, that's a common pattern in some place here, so
I've sprinkled some nolint:govet on it.
Signed-off-by: Stephan Renatus <stephan@styra.com>
This commit adds a utility for explicitly creating cancelable timers, to avoid
possible memory leaks caused by some `<-time.After` timer receives in select
statements never being GC'd properly. This issue is fixed in Go 1.23, but
since we're still on Go 1.21, this will resolve the possibility of leaks in
the mean time.
Signed-off-by: Philip Conrad <philip@chariot-chaser.net>
* Adding support for multiple variables at arbitrary locations in rule refs
* Updating type-checker to handle general ref heads
Fixes: #5993Fixes: #5994
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Default functions satisfy the following properties:
* Same arity as other functions with the same name
* Arguments should only be plain variables ie. no composite values. For ex, default f([x]) = 1 is an invalid default function
* Variable names should not be repeated ie. default f(x, x) = 1 is an invalid default function
Fixes: #2445
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
This commit fixes several memory leaks in the WASM engine that occur
when a caller mixes incremental calls to opa_value_path_add() /
opa_value_path_remove() with actual policy evaluations. The issue
occurs due to a combination of lack of deep free of internal data
structures and the fact that eval() and opa_eval() calls reset the heap
to free temporary memory that they previous allocated.
More details about the issues and their fix are described in detail at
https://github.com/open-policy-agent/opa/issues/5785.
The changes in this patch fall into 5 categories:
1. Adding support for both internal WASM functions and external WASM
callers to perform a "deep" free of OPA values by freeing not only
the immediate object memory but all the opa values it refers to.
The opa_value_free() function now does this by default and is
also an exported function. The opa_value_free_shallow() is added
for the few cases where shallow frees are required, primarily in
eval()-invoked functions.
2. Enable stashing of free blocks prior to eval() and opa_eval() calls.
Eval calls will always leak free blocks due to the way that
opa_heap_ptr_get() works. This patch adds three new exports allowing
the user to save this memory from leaking.
* opa_heap_blocks_stash() -- saves free heap blocks to shadow
freelists.
* opa_heap_blocks_restore() -- restores the allocated heap blocks from
shadow freelists.
* opa_heap_stash_clear() -- discard any saved heap blocks on the shadow
freelists. (this is used for resetting VM heap to an initial state)
3. Update the WASM calling conventions in the SDK. This includes using
the new APIs to avoid leaking memory when adding or removing data
from the data doc. It requires bumping the WASM ABI to 1.3
4. Adding unit tests for the WASM ABI 1.3 functions.
5. Adding documenttion for the WASM ABI 1.3 functions.
Fixes: #5785
Signed-off-by: Chris Telfer <chris.telfer@sophos.com>
Release notes for Wasmtime 3.0.0:
https://github.com/bytecodealliance/wasmtime/blob/main/RELEASES.md#300
But we're also now getting a bunch of CVE fixes, which is the real reason for
updating this dependency:
CVE-2022-39392 - modules may perform out-of-bounds reads/writes when the pooling allocator was configured with memory_pages: 0.
CVE-2022-39393 - data can be leaked between instances when using the pooling allocator.
CVE-2022-39394 - An incorrect Rust signature for the C API wasmtime_trap_code function could lead to an out-of-bounds write of three zero bytes.
We're not using their allocator, but the last one could have been a problem for us, too.
I've never seen it happen in the wild, but it's definitely a code path that we're using.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Got a few warnings from my IDE about redundant type conversions,
so I decided to look into it. Added the unconvert linter to our
checks, and fixed the violations. Added two ignore comments as I
wasn't sure about whether they'd change the semantics of the code.
Signed-off-by: Anders Eknert <anders@eknert.com>
Before, this was OK:
test_a {
. mock_f := true
allow with f as mock_f
}
but this had panicked:
mock_f := true
test_a {
allow with f as mock_f
}
Which, from a user perspective, is quite incomprehensible. Technically,
the first snippet was a (supported) replacement-by-value, and the second
was an unsupported replacement by a rule that was not a function.
Furthermore, the second case wasn't properly caught in the 'with' validations.
Now, we'll capture the situation, and start supporting it. Both snippets will
now work the same, as one would expect from the language surface.
Fixes#5299.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Updated the error wrapping in vm.go as it was looking for an error
message that was no longer present in 0.40.0 (and 1.0.0). It now checks
if the `wasm trap: interrupt` is present in the Trap Message which
I believe is now the correct message to check for.
Signed-off-by: Peter Macdonald <macdonald.peter90@gmail.com>
With the embed directive, we no longer need our custom code that predates
Go 1.16. Also, with the release of 1.19, we no longer desire compatibility
with anything predating 1.16, so this cleanup becomes possible.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This commit includes evaluator support for an opt-in, non-deterministic
builtins caching system, designed to help with future replay of decision
logs.
The cache allows early-exit in the evaluator if the builtin is
non-deterministic, and has already cached a result. Since the cache can
be pre-populated by `rego` module users, this should make offline policy
testing and future work around decision replay more straightforward.
Fixes: #1514
Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
This package is deprecated, archived, and in maintenance mode, since Go
errors support wrapping natively.
For #2152.
Signed-off-by: Jason Hall <jason@chainguard.dev>
* build(deps): bump wasmtime-go: 0.35.0 -> 0.36.0
* internal/wasm: adapt to using epoch-based interruption
Looks like we don't get frames for this.
Also, there is currentlty no better way than comparing the message,
as the trap code isn't surfaced (yet).
Fixes#4663.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Before, we had been planing `every x in xs { BODY }` as, roughly,
NOT
SCAN xs
NOT
BODY
This poses a problem: scans never fail -- they iterate _their_
bodies, and break out of their block when they are done iterating.
As far as the NOT is concerned, it always looks like the SCAN was
successful.
We had worked around that for the "outer" SCAN by breaking out of
the NOT's block at the end. However, the same trick can't be used
when BODY contains another SCAN, since we can't tell beforehand how
deeply we're nested, and doesn't know how far to break out again.
So in this commit, we replace the NOTs by some custom "condition
var" construct that works similar to how NOT gets compiled, but
sets the "inner" condition variable when the BODY's plan succeeds.
Likewise, the "outer" condition variable is tied to that result.
We're practically using the condition variables to encode
every x, y in xs { p(x,y) }
~> p(x1, y1) AND p(x2, y2) AND ... AND p(xn, yn)
~> NOT (NOT p(x1, y1) OR NOT p(x2, y2) OR ... OR NOT p(xn, yn))
The conditon variables are now implemented in the IR. (When using
ir.NotStmt, they are an artifact of compiling the IR to wasm.) This
is achieved by resetting a new local, and assigning it a dummy value
(true) to signal the condition was met.
----
This path was taken because I could not come up with a way to deal
with the "inner SCAN" situation just by using NOT blocks. The "outer"
NOT/SCAN could perhaps be salvaged, but I found it easier to reason
about one mechanism applied twice than to mix and match.
Another abandoned path was using ir.ReturnLocalStmt in the query's
plan iterator: while it worked well for simple cases, the problems
considered were that we might do the wrong thing in nested sitations,
like comprehensions. Also, it's not something we'd done in any other
place.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Due to excessive nesting of blocks, the break statement wasn't breaking
out of enough of them: this would this add an element to the result set
when it should have turned into undefined.
This was the case in two conditions:
a. the dynamic lookup not finding a data function to call, and the resulting
static data lookup failing to resolve the ref; or
b. the dynamically called function returns undefined
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Adding host allow-listing based on the allow_net capability in the http.send()- and
net.lookup_ip_addr() built-in functions when running the eval command.
Fixes: #3665
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Before, we'd either trap with out-of-bounds memory access, or other weird
behaviour.
Included some code cosmetics in pool.go, replacing `for` loop with a `copy`.
Added tests to opa_test.go.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
When using the wasm evaluator, which uses the ABI1.2 `opa_eval` call,
if was possible that the input would exceed the available space after the
heap ptr. A panic would happen.
Now, we'll capture that situation, and grow the memory accordingly
before writing `input` to it.
This is the analogous issue of https://github.com/open-policy-agent/npm-opa-wasm/issues/89.
Thanks to @nichenqin-treelab for reporting.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This sets the stage for eventually allowing OPA wasm modules that do NOT
import memory.
With this change, we add the necessary segments to the wasm module that
declare that memory is to be imported, in the wasm compiler. As far as
LLVM and our C base is concerned, the memory is NOT imported.
The test runners have been adapted, `make wasm-lib-test` works without
having imported memory now. `make wasm-rego-test` can deal with both: it
will provide memory in its `imports` for instantiation, but if the wasm
module happens to not want that import, it'll be ignored. The memory used
in the other host methods is the exported one. (Whether that is exported
or re-exported imported doesn't make a difference.)
Some first steps have been included to make OPA's Wasm SDK work without
imported memory. There are a few loose ends around enforcing memory
limits, to be taken care of later.
----
This also addresses a problem we've seen in the wild before: when our
additions to the wasm modules' data segments exceed the number of pages
needed for the memory import, a "data segment overflowing memory" issue
could have happened. That was because the minimal memory size for the
imported memory was determined by LLVM, and we'd just squeeze our added
data segments in, without adjusting that limit.
Now, the limit will be set properly; and if a too small memory was
provided, a more descriptive failure will happen at instantiation time.
Wasmtime, for example, raises
incompatible import type for `env::memory`
Caused by:
memory types incompatible
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This commit plumbs the print hook through to the wasm runtime so that
print calls are enabled when the wasm target is on. This commit also
updates the planner and compiler to support calls to void
functions--previously, the planner and wasm backend assumed that
functions returned values so they would perform checks for defined
values, however, with void functions, those checks must be suppressed.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
* wasm-sdk: use wasmtime.Linker for resolving imports
No functional change here, but this is easier to read and understand:
anything the module needs, the linker will try to satisfy using the
defined module/name entries ("env.memory", "env.opa_println", etc).
* wasm-sdk: only use one *wasmtime.Engine instance
This should bring us a little more in-line with the
https://docs.rs/wasmtime/0.29.0/wasmtime/#example-architecture
> When the server starts, we’ll start off by creating an Engine (and
> maybe tweaking Config settings if necessary). This Engine will be
> the only engine for the lifetime of the server itself.
> Next, we can compile our WebAssembly.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Stephan Renatus