All packages, except for `cmd` and `internal`, have been moved into a new `v1` root package.
Old packages are kept for backwards-compatibility reasons. All contained code is replaced with simple type aliases and proxy functions to `v1` implementations.
Old packages default to the Rego v0 syntax, new `v1` packages default to the Rego v1 syntax.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
For the first time, down under a 100 million allocations
when running `regal lint bundle` 🎈
**main**
```
BenchmarkLintAllEnabled-10 1 2538350916 ns/op 6182626816 B/op 108424249 allocs/op
```
**pr**
```
BenchmarkLintAllEnabled-10 1 2282894416 ns/op 5310032744 B/op 93674054 allocs/op
```
But there's more to it than just the number of allocations:
```
➜ hyperfine -i --warmup 1 'regal lint bundle' 'regal-new lint bundle'
Benchmark 1: regal lint bundle
Time (mean ± σ): 2.822 s ± 0.055 s [User: 19.299 s, System: 0.603 s]
Range (min … max): 2.743 s … 2.961 s 10 runs
Benchmark 2: regal-new lint bundle
Time (mean ± σ): 2.373 s ± 0.040 s [User: 15.940 s, System: 0.575 s]
Range (min … max): 2.315 s … 2.435 s 10 runs
Summary
regal-new lint bundle ran
1.19 ± 0.03 times faster than regal lint bundle
```
Most notable changes:
- Reuse trieTraversalResult in indexing, as these were expensive
and short-lived. This had the most dramatic impact on the number
of reduced allocations of all the changes here.
- Optimize *set, *object and *Array operations to minimize
allocations by using "primitive" form iteration instead of
the function literal counterparts internally, and to only
reset the sort guard when needed.
- New Array.Equal implementation does not remove any allocations
as the old implementation didn't allocate either. It did however
perform much better for the case where the compared arrays were
not equal.
Signed-off-by: Anders Eknert <anders@styra.com>
# Conflicts:
# topdown/casts.go
- Updated extractNumAndUnit to recognize and correctly parse scientific notation, including cases with 'e' or 'E' followed by an exponent (e.g., "1e10", "3.2E4").
- Ensures that 'e' or 'E' is treated as part of the number if followed by digits, enhancing compatibility with scientific notation inputs.
- Scientific notation with units now also parses correctly (e.g., "1e10GB" extracts "1e10" as the number and "GB" as the unit).
- Maintains behavior for cases without units, allowing either the number or unit portion to be empty, while improving overall string parsing logic.
- Added tests for scientific notation parsing in `units.parse_bytes`, including both SI and binary units (e.g., KB, MiB, GiB, KiB).
- Included cases for uppercase, lowercase, and mixed case formats with scientific notation.
- Added validation for numbers without units defaulting to bytes.
Fixes#7142
Signed-off-by: bakar <berdan.akar@rise-world.com>
This is the last few tests to be refactored before all tests are compatible with the v1-by-default switch coming in OPA 1.0.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Due to insufficient indentation, this file would be rejected by compliant YAML parsers.
This small fix resolves the issue.
Signed-off-by: Matthew Johnson <matjoh@microsoft.com>
In the Wasm modules built with OPA, arithmetic comparison for very large numbers
could be wrong, caused by an integer overflow.
Fixes#6991.
Signed-off-by: Pierre Troger <pierre@narval.xyz>
Fixing issue where ref-head rules could put evaluation result scoped by call-site ref unification into global virtual-cache, which would later erroneously be read by ref to same rule/virtual document but with different "unification scope".
Fixes: #6926
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Fixes https://github.com/open-policy-agent/opa/issues/6925
This change modifies the `io.jwt.encode_sign` built-in to marshal the
parameters as JSON before signing the JWT. The previous implementation
was using String(), which was a sort of pseudo-JSON that was almost
correct but used the wrong JSON syntax for the set type.
I have chosen to marshal the headers in the same way as the payload as
the 'crit' header can also be an array type. I marshal the signature the
same way for consistency with the other two.
Signed-off-by: Charlie Egan <charlie@styra.com>
Fixing an issue where a non-collection `every`-domain didn’t fail evaluation.
Removing a possible attack surface, where an attacker with the ability to craft portions of the input document could replace a value with an expected collection type, that is known to be processed by an `every`-statement, with a non-collection value and thereby would cause the policy to accept a query that should otherwise be rejected.
Fixes: #6762
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Fixing two issues where Early Exit was being suppressed when it shouldn't have been:
1. A cache hit for a rule/function discards EE for the call-site.
2. Non-EE rule/func discards EE for call-site.
Fixes: #6566
To support uses cases where OPA is used for signing s3 requests whose payload is
not known upfront or payload is big enough (big file upload) to be sent over wire,
this PR adds support for unsigned payloads.
AWS signer has configurable option to use unsigned payload where the
x-amz-content-sha256 is set to "UNSIGNED-PAYLOAD" and is included as part
of signing process. This PR provides an option for unsigned payload if
aws_config.disable_payload_signing is set to true. If payload signing is
disabled, SignV4 method will not compute the content sha from the request body
but instead use "UNSIGNED-PAYLOAD" string literal for x-amz-content-sha256
header during signature computation.
References:
https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.htmlhttps://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-auth-using-authorization-header.html
Signed-off-by: Prasanth Jayachandran <p_jayachandran@apple.com>
This change adds a new flag to `opa run` to allow
users to specify a list of enabled TLS 1.0–1.2 cipher
suites. This allows users to control the cipher suites
the OPA server supports during a TLS handshake.
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
This commit adds a config for yamllint, mass-reformats all of
the existing Yaml testcases to pass linting, and adds a Yaml
linting job to the pull-request Github Actions workflow. A few
careful exceptions and ignores were added to the linter's
config to allow keeping our existing Yaml files with minimal
reformatting.
Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
Currently while all the headers are signed the headers in the
returned object are missing all the original user provided headers
This means that if you pass the object directly to http.send
amazon will fail the request because the signed data doesn't match
the canonical request. Users can work around it by using object.union
to restore the original headers, but would be nice to avoid that extra
step
Signed-off-by: Peter <c2zwdjnlcg@users.noreply.github.com>
This is being added to make it easier to write policy on the contents of
certificate URI SANs. This is where information like SPIFFE IDs etc are
contained and it's helpful to Rego authors to have access to these
values without rebuilding the URI from the parsed data under URIs.
Fixes: #6416
Signed-off-by: Charlie Egan <charlie@styra.com>
This adds support for rendering of templated strings utilizing Golang's text/template library.
For a given templated string and key/value mapping of template var inputs, this builtin will
inject the values into the template where they are referenced by key.
Fixes#6371
Signed-off-by: Rohan Vasavada <rohanvasavada@gmail.com>
This way the object insert operations can return a new object instance.
Before, the object construction for a rule like
p[a][b] := ...
would look like this:
*ir.BlockStmt BlockStmt (1 blocks)
*ir.Block Block (3 statements)
*ir.BlockStmt BlockStmt (1 blocks)
*ir.Block Block (2 statements)
*ir.DotStmt &{Source:{Value:Local<2>} Key:{Value:Local<10>} Target:Local<14>}
*ir.BreakStmt &{Index:1}
*ir.MakeObjectStmt &{Target:Local<14>}
*ir.ObjectInsertOnceStmt &{Key:{Value:Local<10>} Value:{Value:Local<14>} Object:Local<2>}
*ir.ObjectInsertOnceStmt &{Key:{Value:Local<11>} Value:{Value:Local<13>} Object:Local<14>}
Now, it'll look like
*ir.BlockStmt BlockStmt (1 blocks)
*ir.Block Block (2 statements)
*ir.BlockStmt BlockStmt (1 blocks)
*ir.Block Block (2 statements)
*ir.DotStmt &{Source:{Value:Local<2>} Key:{Value:Local<10>} Target:Local<14>}
*ir.BreakStmt &{Index:1}
*ir.MakeObjectStmt &{Target:Local<14>}
*ir.ObjectInsertOnceStmt &{Key:{Value:Local<11>} Value:{Value:Local<13>} Object:Local<14>}
*ir.ObjectInsertStmt &{Key:{Value:Local<10>} Value:{Value:Local<14>} Object:Local<2>}
so the object in Local<14> is built first, and the added to object Local<2>.
Signed-off-by: Stephan Renatus <stephan@styra.com>
Co-authored-by: Teemu Koponen <koponen@styra.com>
When feeding a `char *` into `re->Match()`, it was converted to a StringPiece,
taking its size as `strlen()`. For our (long) input, that wasn't resulting in
the correct size, and did then freak out the re2 match input validation if the
regular expression has an end anchor, but the endpos wasn't the same as its
length. Since the endpos was taken from `s->len`, and the "length" taken via
the mentioned StringPiece's strlen() call, they did indeed not match.
Worked around by feeding it a properly-constructed std::string instead. I'm a
C++ novice at best, but it does the trick, and I'm reasonable certain it's less
wrong than before.
Fixes#6376.
Signed-off-by: Stephan Renatus <stephan@styra.com>
We do a lot of `walk`-ing in [Regal](https://docs.styra.com/regal).
So much that it's by far the single most expensive operation. That means
any optimization of the `walk` built-in function will be a win for us.
Seeing as we rarely make use of the `path` component when `walk`-ing
through AST inputs, I was curious to see if there was any optimization
we could take when the path is a wildcard assignment, and as such clearly
marked as unimportant. Turns out there is. Since the return value is
provided in the operators list, we can check the value provided for the
`path` part of the assigned array, and if it's a wildcard (`_`) skip
path construction entirely. Example:
```rego
walk(input, [_, value])
```
This greatly simplifies the walk, and the performance gains are
substantial. Traversing a ~7MB AST:
**main**
```shell
$ opa bench -d p.rego -i objects.json data.p.w
+-------------------------------------------+---------------+
| samples | 6 |
| ns/op | 168806625 |
| B/op | 197364318 |
| allocs/op | 3855327 |
| histogram_timer_rego_query_eval_ns_75% | 169968114 |
| histogram_timer_rego_query_eval_ns_90% | 170513459 |
| histogram_timer_rego_query_eval_ns_95% | 170513459 |
| histogram_timer_rego_query_eval_ns_99% | 170513459 |
| histogram_timer_rego_query_eval_ns_99.9% | 170513459 |
| histogram_timer_rego_query_eval_ns_99.99% | 170513459 |
| histogram_timer_rego_query_eval_ns_count | 6.00 |
| histogram_timer_rego_query_eval_ns_max | 170513459 |
| histogram_timer_rego_query_eval_ns_mean | 168789611 |
| histogram_timer_rego_query_eval_ns_median | 168924020 |
| histogram_timer_rego_query_eval_ns_min | 166685000 |
| histogram_timer_rego_query_eval_ns_stddev | 1239390 |
+-------------------------------------------+---------------+
```
**no-path-walk**
```shell
$ opa bench -d p.rego -i objects.json data.p.w
+-------------------------------------------+--------------+
| samples | 21 |
| ns/op | 50629984 |
| B/op | 38018790 |
| allocs/op | 1025211 |
| histogram_timer_rego_query_eval_ns_75% | 51239562 |
| histogram_timer_rego_query_eval_ns_90% | 51540933 |
| histogram_timer_rego_query_eval_ns_95% | 51674420 |
| histogram_timer_rego_query_eval_ns_99% | 51688208 |
| histogram_timer_rego_query_eval_ns_99.9% | 51688208 |
| histogram_timer_rego_query_eval_ns_99.99% | 51688208 |
| histogram_timer_rego_query_eval_ns_count | 21.0 |
| histogram_timer_rego_query_eval_ns_max | 51688208 |
| histogram_timer_rego_query_eval_ns_mean | 50611103 |
| histogram_timer_rego_query_eval_ns_median | 50871459 |
| histogram_timer_rego_query_eval_ns_min | 49518833 |
| histogram_timer_rego_query_eval_ns_stddev | 748688 |
+-------------------------------------------+--------------+
```
The real-world impact is not as dramatic, since we aren't *just*
walking, but normally need to actually **do** something with the
values returned, but consistently shaving off about 13% eval time
when linting one of the largest policy libraries isn't bad at all:
**Regal main**
```shell
go run main.go lint ~/tmp/kics/assets 162.16s user 6.04s system 593% cpu 28.362 total
```
**Regal walk-no-path**
```shell
go run main.go lint ~/tmp/kics/assets 145.51s user 5.01s system 597% cpu 25.176 total
```
Signed-off-by: Anders Eknert <anders@eknert.com>
* Adding support for multiple variables at arbitrary locations in rule refs
* Updating type-checker to handle general ref heads
Fixes: #5993Fixes: #5994
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
* ast: Including "child" rules when fetching rules by ref
In order to properly sort rules according to dependencies before performing type-checking.
Also, properly merging types when type-tree produces multiple types for a given ref.
Fixes: #6182
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
... instead of static portion.
This change allows object unification/comparison between an object composed at eval-time (i.e. constructed by rules) and a static object that doesn't contain all keys declared by rules, as those might not be defined at eval-time.
Fixes: #6138
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Default functions satisfy the following properties:
* Same arity as other functions with the same name
* Arguments should only be plain variables ie. no composite values. For ex, default f([x]) = 1 is an invalid default function
* Variable names should not be repeated ie. default f(x, x) = 1 is an invalid default function
Fixes: #2445
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
* crypto.parse_private_keys parses private keys, returns a list of valid
keys
* consolidated getPrivateKeysFromString with getPrivateKeysFromPEMData
for crypto.x509.parse_rsa_private_key and crypto.parse_private_keys
* reworked crypto.x509.parse_rsa_private_key so that we no longer need
getPrivateKeysFromString but instead determine type of key in builtin
function and added input validation checks
Co-authored-by: Charlie Egan <git@charlieegan3.com>
Signed-off-by: Emil Volckmar Ry <emilvry@gmail.com>
When evaluating arithmetic operations, numbers are converted from
json.Number to big.Float and back again. Currently the conversion back
to json.Number uses the big.Float Text() method with the 'g' format.
This causes integers with 7 or more digits to be represented with a
decimal point and exponent.
This can be problematic when creating JWT tokens with "iat" and "exp"
timestamps, as apparently some JWT parsing libraries do not handle
NumericDate values formatted with decimal point and exponent.
Instead, modify the FloatToNumber() conversion method to avoid the
exponent format for numbers which are integers.
Fixes#6013.
Signed-off-by: Kenneth Jenkins <kjenkins@pomerium.com>
Anders Eknert