Expanding the Rego syntax to support not-bodies (not blocks?): `not {...}`
For a not block to successfully evaluate, its body must not successfully evaluate. If evaluation causes iteration, all evaluation paths must fail.
Fixes: #8402
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Parsing is generally fast, so this mainly improves performance
of creating big bundles with many Rego files in them. For Regal's
embedded bundle, loading it from memory would previously take 16
ms on my laptop, and now it takes 9 ms. There are other things
in this process that could be concurrent too, like JSON unmarshalling
of multiple data files. But starting with parsing modules.
This PR adds `errgroup` as a direct dependency (previously indirect)
as it is a nicer way to work with wait groups, and one that can be
useful elsewhere in the codebase (like in the compiler).
Also, and as usual, went off on a bit of a tangent refactoring code
related to the bundle build process, and made sure to use some common
helpers in code where available.
Signed-off-by: Anders Eknert <anders.eknert@apple.com>
Funnily, this started out as an attempt to look into issues reported
with compiling large policy sets... before I realized that it isn't
likely *this* compiler that has perf issues, but the one that "compiles"
bundles as part of activation. So while these fixes likely does little
to address that, there are still some rather nice improvements here, where
the big ones as ususal are mostly just wins from avoiding work where it's
possible.
For benchmarking I've used Regal's embedded bundle, which isn't great to
use over time, as it's a moving target. But since it's a pretty extensive
bundle and one that covers most features of OPA, it's at least good for
1:1 comparisons when testing perf improvements.
```
// 66555594 ns/op 50239492 B/op 1083664 allocs/op - main
// 62569440 ns/op 38723015 B/op 944277 allocs/op - compiler-optimizations pr
```
The B/op / alloc_space improvement is particularly nice here. What's noteworthy
is how relatively little impact that has on performance in this case. That may
be surprising but aligns pretty well with my previous experience of Go code where
a lot of time is spend in recursive walks — that simply takes time, no matter how
much you optimize. Oh well, less memory allocated for this is more memory to spend
elsewhere.
(I'm adding the benchmark used below to Regal in a parallel PR)
Signed-off-by: Anders Eknert <anders@styra.com>
This commit adds support for changing out how bundle storage and
activation work. To allow swapping out bundle activation, two new
`bundle` package functions are provided:
- `RegisterActivator`: Registers a bundle.Activator with a string ID.
- `RegisterDefaultBundleActivator`: Sets the default bundle.Activator to
use by ID.
Behind the scenes, a few new `bundle` package variables are used to
track what bundle activators are available, and which is the preferred
default.
This system allows registering many activators, and allows choosing the
bundle activator to use at activation time. The activator to use is
decided in the following order:
- `(bundle.ActivateOpts).Plugin` is used when non-nil.
- `bundle.bundleExtActivator` is used when an ID was set with
`RegisterDefaultBundleActivator`.
- The default/original bundle activator is used if no other selection
was made.
To support swapping out bundle storage (useful when testing new bundle
designs), a new `bundle` package function is provided:
- `RegisterStoreFunc`: Sets the function to use for creating bundle
storage.
These two features together allow swapping out most of the bundle
activation flow, without requiring deep modification of the `bundle`
package. Lazy bundle loading mode is also enabled across many CLI
commands and other bundle loading points now when a non-default bundle
activator is set.
Signed-off-by: Philip Conrad <philip@chariot-chaser.net>
Co-authored-by: Ashutosh Narkar <anarkar4387@gmail.com>
Following up on #7566, and now applying the more exciting
modernizations. fmt.Appendf was new to me! But especially
the contains checks are so much better IMHO. I have reviewed
all changes myself and did a few manual changes where it
became obvious that things could be improved a little further.
(the modernize analyzer still has some issues running against
OPA, and I have manually worked around those for the time being)
Signed-off-by: Anders Eknert <anders@styra.com>
to allow for using Rego v1 bundles in `opa build`/`check`/`eval`/`test`.
Before this change, a bundle with `1` as `rego_version`/`file_rego_versions` would be rejected when evaluated with the `--v0-compatible` flag with the error:
```
rego_parse_error: illegal capabilities: rego_v1 feature required for parsing v1 Rego
```
This is fixed by adding the `rego_v1` feature to the `v0` default capabilities applied when using the `--v0-compatible` flag. Note: this allows OPA to accept Rego `v1` modules inside bundles, but modules without a specified Rego version, such as freestanding non-bundle modules or modules inside bundles with no specified Rego version, are parsed as `v0`.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
where the bundle has been manually constructed containing v0 Rego modules and
no `rego_version`/`file_rego_versions` fields are declared in the bundle manifest.
This affects bundle deactivation in the bundle store lifecycle used when for
`opa run` in server mode (`-s`).
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
And update code to conform to the rule.
- Replace unnecessary fmt.Sprintf with string concatenation
- Replace fmt.Sprint with more efficient strconv.Itoa
- Replace static fmt.Errorf calls with more efficient errors.New
Thanks @srenatus for pushing me down this rabbit hole!
Signed-off-by: Anders Eknert <anders@styra.com>
This was originally added to allow Regal to get a serialized AST that included
location data wherever that was possible. Regal is however no longer using OPA's
JSON serialization but its own custom encoder. Attaching these options to every
AST node and term comes with a cost attached, and without any known users of this
feature vs. the many users who care about resource utilization, this feels like
an easy choice.
While it seems unlikely to be users depending on this functionality — in case
someone needs it, the options for serializing AST nodes to JSON can now be set
globally instead. Global state is always awkward, but since JSON marshalling
methods only have access to the node being marshalled and of course, global
state, there's not a whole lot of options if we intend to keep this feature.
Signed-off-by: Anders Eknert <anders@styra.com>
Fixing an issue where the rego-version for individual modules was lost during bundle deactivation (bundle lifecycle) if this version diverged from the active runtime rego-version. This could cause reloading of v0 bundles to fail when OPA was not running with the `--v0-compatible` flag.
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
Co-authored-by: Johan Fylling <johan.dev@fylling.se>
All packages, except for `cmd` and `internal`, have been moved into a new `v1` root package.
Old packages are kept for backwards-compatibility reasons. All contained code is replaced with simple type aliases and proxy functions to `v1` implementations.
Old packages default to the Rego v0 syntax, new `v1` packages default to the Rego v1 syntax.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Johan Fylling