All packages, except for `cmd` and `internal`, have been moved into a new `v1` root package.
Old packages are kept for backwards-compatibility reasons. All contained code is replaced with simple type aliases and proxy functions to `v1` implementations.
Old packages default to the Rego v0 syntax, new `v1` packages default to the Rego v1 syntax.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Before, any call to a builtin that would reference an object from data would force that
the entire object was transformed to an ast.Value, and a copy of it was passed to the builtin.
Now, we're doing something a little more involved: when an object is read from data, we'll
wrap it into a special container and don't convert it. The conversion only happens when it's
required. As such, we can avoid quite a bit of memory usage, and avoiding the extra work,
make evaluations faster.
Lazy objects are also immutable -- calling Insert() on them will panic. But based on how
topdown works, it should never happen. Only Golang coding modifying the results of a query
evaluation could be affected by this. If you are, see the opt-out mechanisms outlined below.
⚠️ The benchmark that follows is an edge case, but the issue at hand here has come up in
the real world before.
name old time/op new time/op delta
MemberWithKeyFromBaseDoc-16 40.2ms ±13% 0.0ms ± 5% -99.97% (p=0.000 n=9+10)
ObjectGetFromBaseDoc-16 42.4ms ± 4% 0.0ms ± 2% -99.97% (p=0.000 n=9+10)
name old alloc/op new alloc/op delta
MemberWithKeyFromBaseDoc-16 17.2MB ± 0% 0.0MB ± 0% -99.96% (p=0.000 n=10+10)
ObjectGetFromBaseDoc-16 17.2MB ± 0% 0.0MB ± 0% -99.95% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
MemberWithKeyFromBaseDoc-16 702k ± 0% 0k ± 0% -99.98% (p=0.000 n=10+10)
ObjectGetFromBaseDoc-16 702k ± 0% 0k ± 0% -99.98% (p=0.000 n=10+10)
Note: The optimization is enabled by default, and for all storage implementations. However,
there are multiple layers of opt-out options, in case this causes any unforeseen trouble for
your usage of OPA via Golang. (OPA-as-server use cases don't need to worry about this.)
Concretely, these opt-out options are:
1. an option for `topdown.Query` to not start with lazy objects in the first place
2. an option for `ast.JSONWithOpt` to revert to the old behaviour when converting to golang native values
3. an `EvalOption` for `rego.PrepareForEval(...).Eval(...)` so that it keeps copying the maps it adds to the result set
Fixes#5325.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Previously we had constants defined for AST type names. These were used
in error messages in various places. The original goal was to make error
messages consistent, however, this approach made it difficult to locate
the source of the error in code.
These changes add separate infix operators for assignment and equals
(from equality/unification which was previously used for all three.)
With assignment, authors can declare local variables that will shadow
globals.
These changes allow calls to be nested inside terms (e.g., f(x) !=
g(x)). As part of these changes a few things have been refactored:
1) Grammar has been restructured so that construction code is pulled out
into a separate file. Hopefully this makes the grammar more readable.
2) String() implementation on Expr has been simplified to use prefix
notation for calls (except equality) as this avoids the challenge of
worrying about roundtripping policy strings (which is done frequently
inside test cases.) E.g., plus(x,1,y) converted to infix x + 1 = y would
parse to eq(x + 1, y).
Previously, functions were implemented with a separate set of types that
had their own code paths in the compiler, eval, etc. These changes
refactor the function implementation so that functions are implemented
as rules with one or more arguments.
By representing functions as rules, we can avoid special casing required
to support functions, e.g., during parse and compile there are a number
of steps that required special casing for functions:
- Parser needed separate grammar definitions for functions (which
prevented them from being chained or using else)
- Compiler needed separate resolver and type checker implementations
which was a source of bugs.
In some cases, special casing is unavoidable for now (e.g., during eval)
however this could be improved in the future.
Fixes#471Fixes#467Fixes#463
Anders Eknert