Fixing an issue where the rego-version for individual modules was lost during bundle deactivation (bundle lifecycle) if this version diverged from the active runtime rego-version. This could cause reloading of v0 bundles to fail when OPA was not running with the `--v0-compatible` flag.
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
Co-authored-by: Johan Fylling <johan.dev@fylling.se>
(cherry picked from commit 5d5329dffb)
This changes updates the docs and all the policy examples in them to
be OPA v1.0-compliant. It also binds the OPA server to `localhost`
interface by default per OPA v1.0 specs.
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
Co-authored-by: Charlie Egan <charlie@styra.com>
My last PR for a while in the ongoing "reduce allocations in eval" quest.
Motivated initially mostly to speed up `regal lint`, but most of the changes
here positively impacts evaluation performance for most policies.
The changes with the highest impact in this PR:
* Use `sync.Pool`s to avoid the most costly allocations, includuing heavy `*eval`
pointers created each time a child or closure scope is evaluated.
* When tracing is disabled, avoid variable escaping to heap in `evalStep` function
whose value is only read when tracing is enabled.
* Save one allocation per iteration in `walkNoPath` by reusing an AST array instead
of creating a new one for each call.
Also a few minor fixes here and there which either fixed some correctness issue, or
had a measurable (although minor) positive impact on performance.
**regal lint bundle (main)**
```
BenchmarkRegalLintingItself-10 1 2015560750 ns/op 4335625360 B/op 83728460 allocs/op
```
**regal lint bundle (now)**
```
BenchmarkRegalLintingItself-10 1 1828754125 ns/op 3541027496 B/op 70080568 allocs/op
```
About 10% faster eval, with almost a gigabyte less memory allocated, and 13 million+ allocations
less performed.
Another topic discussed recently has been the cost of calling custom functions in hot paths.
While this PR doesn't address that problem fully, the benefits of the change is still quite
noticeable. A benchmark for that case specifically is also included in the PR, and the change
compared to main as noted below:
**main**
```
BenchmarkCustomFunctionInHotPath-10 55 18543908 ns/op 20821043 B/op 284611 allocs/op
```
**pr**
```
BenchmarkCustomFunctionInHotPath-10 73 16247587 ns/op 13048108 B/op 228406 allocs/op
```
It's worth noting however that this benchmark benefits "unfairly" by the improvements made
in the `walkNoPath` function, and perhaps more so than custom function evaluation getting
that much more efficient.
Signed-off-by: Anders Eknert <anders@styra.com>
When an OPA instance runs for a long time, it seems odd to send version reports
every hour. I think it's unlikely that someone watches the logs at that point.
So this change makes OPA report every 6 hours (plus a random time between 0 and
60 minutes), after it has reported hourly (+spray) for 6 times.
Signed-off-by: Stephan Renatus <stephan@styra.com>
to maximize compatibility surface across OPA versions.
Adding `--drop-v0-imports` flag to `opa fmt` for opting in to dropping redundant v0 imports.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
All packages, except for `cmd` and `internal`, have been moved into a new `v1` root package.
Old packages are kept for backwards-compatibility reasons. All contained code is replaced with simple type aliases and proxy functions to `v1` implementations.
Old packages default to the Rego v0 syntax, new `v1` packages default to the Rego v1 syntax.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
The topdown Cancel machinery is there because it's cheap to check. ctx.Err() is
not.
This change moves the "Is the context the cause for cancellation?" check into
the branch were evaluation has been aborted through the topdown.Cancel call.
When evaluation has already been cancelled, an expensive check no longer
matters much -- when it's still ongoing, it'll affect the overall performance.
Signed-off-by: Stephan Renatus <stephan@styra.com>
Per discussion in https://github.com/open-policy-agent/opa/pull/7188, do *not* adjust the timing of cache scans to be more frequent; we want to wait at least staleEntryEvictionTimePeriodSeconds between cache scans, even if we hold the lock for a substantial period of time.
Signed-off-by: Evan Anderson <evan@stacklok.com>
For the first time, down under a 100 million allocations
when running `regal lint bundle` 🎈
**main**
```
BenchmarkLintAllEnabled-10 1 2538350916 ns/op 6182626816 B/op 108424249 allocs/op
```
**pr**
```
BenchmarkLintAllEnabled-10 1 2282894416 ns/op 5310032744 B/op 93674054 allocs/op
```
But there's more to it than just the number of allocations:
```
➜ hyperfine -i --warmup 1 'regal lint bundle' 'regal-new lint bundle'
Benchmark 1: regal lint bundle
Time (mean ± σ): 2.822 s ± 0.055 s [User: 19.299 s, System: 0.603 s]
Range (min … max): 2.743 s … 2.961 s 10 runs
Benchmark 2: regal-new lint bundle
Time (mean ± σ): 2.373 s ± 0.040 s [User: 15.940 s, System: 0.575 s]
Range (min … max): 2.315 s … 2.435 s 10 runs
Summary
regal-new lint bundle ran
1.19 ± 0.03 times faster than regal lint bundle
```
Most notable changes:
- Reuse trieTraversalResult in indexing, as these were expensive
and short-lived. This had the most dramatic impact on the number
of reduced allocations of all the changes here.
- Optimize *set, *object and *Array operations to minimize
allocations by using "primitive" form iteration instead of
the function literal counterparts internally, and to only
reset the sort guard when needed.
- New Array.Equal implementation does not remove any allocations
as the old implementation didn't allocate either. It did however
perform much better for the case where the compared arrays were
not equal.
Signed-off-by: Anders Eknert <anders@styra.com>
# Conflicts:
# topdown/casts.go
This changes the `eval_cancel_error` logic to differentiate between
context.Canceled and context.DeadlineExceeded errors, because they are
distinct errors. By differentiating here, callers will be able to
determine if a call was stopped because a caller stopped the request, or
because the request went over some imposed deadline.
Signed-off-by: Mike Chittenden <mchittenden@digitalocean.com>
This isn't needed anymore, so now we don't.
Also enabled the copyloopvar linter in case we
accidentally do this in the future.
Signed-off-by: Anders Eknert <anders@styra.com>
**main**
```
BenchmarkLintAllEnabled-10 1 2640715625 ns/op 6385110200 B/op 116296633 allocs/op
```
**pr**
```
BenchmarkLintAllEnabled-10 1 2597179708 ns/op 6183614112 B/op 108421141 allocs/op
```
(I renamed the benchmark, but this is the same as "regal linting itself"
used in the past)
Another 8 million allocations cut off from `regal lint bundle`,
and a whopping 10% improvements to wall clock time!
The most significant improvement is the Equal implementation for
refs, since that is called all over the place. But there are many
other fixes here, and they all contribute something substantial
(and fixes that only have had marginal impact have been left out).
Signed-off-by: Anders Eknert <anders@styra.com>
Previously, this would panic deep win the call stack when trying to do the actual
partial eval. Now, we're parsing the arguments and decline bad input with an error
message.
Fixes#7127.
Signed-off-by: kunal.das <kd.kunaldas92@gmail.com>
* Respect runtime rego-version in RESTful policy API
Updating `/v1/policies` RESTful server endpoint to respect the `--v0-compatible`/`--v1-compatible` flags.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* Debugger: allow YAML to be used as input
By using the unmarshal function from util instead of the stdlib
Signed-off-by: Anders Eknert <anders@styra.com>
* Adding test
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
---------
Signed-off-by: Anders Eknert <anders@styra.com>
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Co-authored-by: Johan Fylling <johan.dev@fylling.se>
Johan Fylling