All packages, except for `cmd` and `internal`, have been moved into a new `v1` root package.
Old packages are kept for backwards-compatibility reasons. All contained code is replaced with simple type aliases and proxy functions to `v1` implementations.
Old packages default to the Rego v0 syntax, new `v1` packages default to the Rego v1 syntax.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
**main**
```
BenchmarkLintAllEnabled-10 1 2640715625 ns/op 6385110200 B/op 116296633 allocs/op
```
**pr**
```
BenchmarkLintAllEnabled-10 1 2597179708 ns/op 6183614112 B/op 108421141 allocs/op
```
(I renamed the benchmark, but this is the same as "regal linting itself"
used in the past)
Another 8 million allocations cut off from `regal lint bundle`,
and a whopping 10% improvements to wall clock time!
The most significant improvement is the Equal implementation for
refs, since that is called all over the place. But there are many
other fixes here, and they all contribute something substantial
(and fixes that only have had marginal impact have been left out).
Signed-off-by: Anders Eknert <anders@styra.com>
This commit swaps out the old algorithm in the `(Any).Union` method that
would explode on some inputs to one that uses dramatically less memory,
and runs in nearly linear time and memory usage.
This commit also includes improvements to the
`BenchmarkCompileDynamicPolicy` benchmark, as well as new benchmarks for
the `Any` type's `Merge` and `Union` methods.
Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
* Adding support for multiple variables at arbitrary locations in rule refs
* Updating type-checker to handle general ref heads
Fixes: #5993Fixes: #5994
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* ast: Including "child" rules when fetching rules by ref
In order to properly sort rules according to dependencies before performing type-checking.
Also, properly merging types when type-tree produces multiple types for a given ref.
Fixes: #6182
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* compile/compile_bench_test: Add basic compilation benchmark.
* types/types: Replace linear copy loops with copy() Go builtin.
* types/types: Switch linear scan -> binary search.
This commit switches out the linear scan in the `Any` type's `Contains`
method for a more efficient binary search. This results in around a 3-5x
speedup for policy compilation as a whole, according to the benchmark.
* ast/visit: Remove for loop item copies.
This commit refactors the visitor functions in the `ast/visit` package
to avoid extra copying of items on each loop iteration. This shaves off
around 10-15% memory usage during type-checking during compilation, and
provides speed benefits due to reduced GC as a result.
Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
This commit adds the `prealloc` linter to the list of linters for OPA, and fixes up the miscellaneous locations in the code that the linter found where we could easily preallocate slices.
Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
This commit adds support for named argument declarations for built-in
functions as well as additional metadata/annotations on built-in
functions (e.g., descriptions, categories, etc.) This commit allows us
to generate a data file (builtin_metadata.json) that other tools can
consume to improve the Rego authoring experience.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Co-authored-by: Torin Sandall <torinsandall@gmail.com>
This commit updates the Arity function to return zero if the type is
known and -1 if the type is unknown. This makes the Arity function
behave the same as the GetArity function on the compiler which was
used for similar purposes.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This commit prevents the rewriting step from attempting to capture the
result of void function calls. This avoids generating invalid queries
that will fail to type check.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This commit adds initial support for varargs on function
declarations. Varargs are only supported on void functions for
now. In the future, varargs can be supported on any function, but care
will have to be take in the compiler to handle calls that make use of
the legacy calling convention for functions where the output is
captured in the argument list (e.g., plus(1,2,3)).
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
* types: Sort any elements during construction
This changes updates the implementation of the any type to sort
elements during construction. This way the Compare() function does not
have to sort elements before recursing (which can result in data races
if global type instances from the built-in function declarations or
elsewhere are compared.)
Fixes#3793
* capabilities.json: changed ordering
* internal/presentation: fix json error output
Co-authored-by: Torin Sandall <torinsandall@gmail.com>
Co-authored-by: Stephan Renatus <stephan.renatus@gmail.com>
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
The static properties are stored in sorted order so perform a binary
search to lookup instead of scanning all keys.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
golint is deprecated. The author of the code no longer supports the
codebase. golangci-lint is faster than golint, and is in use by other
opa repositories (e.g. Gatekeeper).
This commit changes tools.go to reference golangci (so it ends up in
vendor) and modifies check-lint to use golangci instead.
Breaking API Changes:
- plugins/rest/rest.go: Fix typo "AllowInsureTLS" -> "AllowInsecureTLS"
- storage/errors.go: Removed unused IndexingNotSupportedErr
Signed-off-by: Will Beason <willbeason@google.com>
Also, there is new support for adding annotations on Rules to specify the schemas to be used specifically for type checking the expressions within the scope of that Rule. It helps address issues with schema overloading, and provides even more precise type error reports for a Rego developer.
Also, added support for annotation processing when loading via bundles.
Co-authored-by: @vazirim Mandana Vaziri mvaziri@us.ibm.com
Co-authored-by: @aavarghese Ansu Varghese avarghese@us.ibm.com
Co-authored-by: @tsandall Torin Sandall torinsandall@gmail.com
Signed-off-by: Mandana Vaziri <mvaziri@us.ibm.com>
The type checker was panicing when given a reference to an object with
a composite key because internally it infers the key type using
ast.ValueToInterface (which returns map[string]interface{} and not
map[interface{}]interface{}). This change updates the types package to
support map[string]interface{} internally.
Fixing the TypeOf function resolved the panic but it surfaced another
issue in the type checker where errors were returned if objects or
arrays were dereferenced with composite keys--in case of arrays, the
result is undefined but in the case of objects, this is valid.
Fixes#2648
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
If a negative array index was hardcoded in the policy it would cause a
panic (e.g., arr[-1]). This patch just fixes the select function to
return nil like it does for out-of-bounds.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously, if callers omitted output terms from call expressions, the
result would be ignored. This was fine for most calls which would only
return true if they were defined, however, for functions could return
false this became confusing because an expression like "f(1)" where f(1)
= false would be succeed and yield a result.
With these changes, built-in functions that used to only return true
always return true or false and the eval engine takes care to check if
the result is false when the the caller omits the output term.
This provides consistent behaviour across cases like...
f(1) => undefined (previously {})
f(1,x) => {x:false} (previously {x:false})
neq(1,1,x) => {x:false} (previously undefined)
neq(1,1,false) => {} (previously undefined)
neq(1,1,true) => undefined (previously undefined)
In the next set of changes, the Rego package will be updated to capture
values for expressions like the first one above so that function calls
behave like refs (i.e., their values are returned).
These changes modify topdown evaluation to use a binding list that
namespaces variables. This allows topdown to propagate partially ground
ref operands into child query evaluation.
These changes also prepare topdown evaluation to support a partial
evaluation mode.
With these changes, evaluation is no longer performed in two steps
(i.e., first pass of evaluating individual terms, second pass of
evaluating built-in expressions.) Instead, evaluation assumes queries
have been rewritten to eagerly evaluate refs and comprehension. This
way, ref and comprehension bindings do not have to be maintained
separately: they are handled by the normal variable binding list.
This commit contains some breaking changes to the topdown APIs,
namely...
1. Truth explanation has been removed. This feature was not used and the
tracing changes broke it. We can revisit in future if necessary.
2. Data indexing has been removed. Data indexing can be re-added in
future if necessary however it should be handled outside of topdown to
avoid potential memory leaks.
3. Built-in functions produce at-most-one output now. Functions that
used to produce multiple outputs (e.g., io.jwt.decode) can produce a
composite value if they need to.
Fixes#131
These changes introduce functions into the type layer. With functions in
the type layer, we can begin treating functions as a first-class
construct within OPA.
The types package defines the data types in OPA. The types themselves
are comparable. The Any type represents a set of types for cases where a
static value cannot be assigned. An empty Any type is the superset of
all types (and sets of types).
Anders Eknert