* crypto.parse_private_keys parses private keys, returns a list of valid
keys
* consolidated getPrivateKeysFromString with getPrivateKeysFromPEMData
for crypto.x509.parse_rsa_private_key and crypto.parse_private_keys
* reworked crypto.x509.parse_rsa_private_key so that we no longer need
getPrivateKeysFromString but instead determine type of key in builtin
function and added input validation checks
Co-authored-by: Charlie Egan <git@charlieegan3.com>
Signed-off-by: Emil Volckmar Ry <emilvry@gmail.com>
crypto.x509.parse_keypair(cert, key) takes PEM/DER (base64) encoded input.
if key pair is valid, returns the tls.certificate(https://pkg.go.dev/crypto/tls#Certificate) as an object.
if key pair is invalid, returns an error.
Fixes#5853
Signed-off-by: Emil Volckmar Ry <emilvry@gmail.com>
Add crypto.hmac.equal built-in function to safelly comparing hashes generated by MD5, SHA-1, SHA-256 and SHA-512 hashing algorithms.
The built-in function is a wrapped for the function Equal of package crypto/hmac.
It's useful when you need to write a policy that requires to check hash signature for request body.
Signed-off-by: Sandokan D. Arantes <sandokandias@gmail.com>
Adds a builtin time.format to format time given in ns to a string timestamp for
the given timezone or UTC as default. The builtin takes in 3 types of arguments:
1. An integer value representing the time in nanoseconds since epoch
2. An array with the first value as integer representing the time in ns and second argument as string value representing the timezone. In the first case, when only an integer value is provided, UTC timezone is considered
The function returns a string type value of the timestamp in the RFC3339Nano format. E.g. 2022-11-23T18:20:14Z
3. An array with the first value the integer ns, the second a string timezone, and third a string for the format, to use one different from RFC3339Nano.
Signed-off-by: burnerlee <avi.aviral140@gmail.com>
This commit adds initial support for AWS's SigV4 request signing system,
which will allow OPA's existing `http.send` builtin to be used to more
conveniently query cloud resources. It automates away most of the pain
around signing the request headers and body, and is designed to compose
with `http.send` directly.
Internally, this also refactors AWS SigV4 request signing, so that the
signing logic is shared between the builtin and the REST plugin for AWS.
Fixes: #3749
Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
The `object.keys` function will return a set of all top-level keys on
a given object. Since object keys in Rego don't have the same
restrictions as names in JSON name-value pairs, we also ensure
support for non-string key types.
Fixes#5363.
Signed-off-by: Kevin Swiber <kswiber@gmail.com>
This commit adds a new GraphQL builtin for validating GraphQL schemas,
applying stronger validation rules than what the current GraphQL parsing
builtins apply by default.
Fixes: #5125
Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
This commit adds the `net.cidr_is_valid` builtin, which makes
validating network CIDR strings much easier in policies.
Example policy:
allow {
net.cidr_is_valid("192.168.0.0/24")
}
This builtin works for both IPv4 and IPv6 CIDR strings.
Signed-off-by: Ricardo Maraschini <ricardo.maraschini@gmail.com>
This commit adds support for AST objects to be usable in place of
strings for several of the GraphQL built-in functions, to improve
the composability of the GraphQL set of built-ins, and to dramatically
reduce the amount of redundant parsing when writing GraphQL policies.
Fixes: #4742
Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
This commit includes evaluator support for an opt-in, non-deterministic
builtins caching system, designed to help with future replay of decision
logs.
The cache allows early-exit in the evaluator if the builtin is
non-deterministic, and has already cached a result. Since the cache can
be pre-populated by `rego` module users, this should make offline policy
testing and future work around decision replay more straightforward.
Fixes: #1514
Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
Using null for delimiters disables delimiters in glob matching. Preferable over regex on some cases for performance reasons.
Fixes#4923.
Signed-off-by: vinhph0906 <vinhph0906@gmail.com>
Co-authored-by: Stephan Renatus <stephan.renatus@gmail.com>
`contains` provides an alternative way to declare partial sets:
p contains x {
x := { "foo": "bar"
}
which is the same as
p[x] {
x := { "foo": "bar"
}
The keyword is enabled by importing `future.keywords.contains`, and
when it _is enabled_, the format will be used for all partial sets in
that file for pretty-printing.
`if` is a new keyword allowing for more readable rule definitions:
The syntax is
NAME [if] { EXPR [EXPR...] }
and the is a shorthand allows dropping the braces around the expression
if there is only one:
NAME if EXPR
For example, this allows expressions like
allow if not deny
f(xs) if every x in xs { x != "foo" }
The one exception here are partial sets: they cannot use `if` UNLESS
they use `contains`:
p[x] { x := "foo" } # valid
p contains x { x := "bar" } # valid
p contains x if { x := "bar" } # valid
p[x] if { x := "foo" } # invalid
This is because we want to interpret that differently (as an object
rule defining `p.foo = true`) in the near future.
The formatter works in the same way: if `future.keywords.if` is imported, it
will be used where it can be used.
We don't want to be too eager when it comes to introducing syntactic sugar.
So this will be rewritten, because head and body expression are on the same
line:
p := 5 if { time.day_of_week() == "Monday" }
# => p := 5 if time.day_of_week() == "Monday"
but this won't:
p := 5 if {
time.day_of_week() == "Monday"
}
The rationale here is that if the policy author decided that they want this on
an extra line, we won't mess with it.
This also sidesteps the need to check if both the head and the single body
expression have a comment.
This change includes various docs updates. Notable exceptions are the GK docs,
since it will take a while for these keywords to be come available there; and
the frontpage: merging a PR would update the frontpage immediately, and we
don't want to show something there that isn't available in the latest release.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This implements the new object.subset() builtin.
Based on my benchmarking, this offers a 2.77x speedup compared to
implementing the same thing in pure Rego, and is also easier to read.
Fixes#4358
Signed-off-by: Charles Daniels <charles@styra.com>
This commit vendors in github.com/vektah/gqlparser, and provides the
implementation for the following graphql built-ins:
- graphql.parse
- graphql.parse_and_verify
- graphql.parse_query
- graphql.parse_schema
- graphql.is_valid
The test suite is comprised of the classic "Star Wars" examples from the
official GraphQL docs, along with a host of syntax examples ported over
from the underlying GraphQL parser's test suite.
AST objects returned by the parse_x APIs are currently pruned for brevity.
Fixes#4283
Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
This function works on all base decimal and binary SI units of the set:
m, K/Ki, M/Mi, G/Gi, T/Ti, P/Pi, and E/Ei
Note: Unlike `units.parse_bytes`, this function is case sensitive.
Fixes open-policy-agent#1802.
Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
New functions:
* rego.metadata.chain(): returns the chain of metadata, starting from the active rule, going outward
* rego.metadata.rule(): returns the metadata for the active rule
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Importing `future.keywords.every` will ALSO import `future.keywords.in`,
since the latter is required for the former.
This includes the formatting of the expression itself, and adding
the "future.keywors.every" import if necessary:
This would happen when pretty-printing an AST that was parsed
with ast.ParserOptions enabling the required future keyword:
The import would not be present in the *ast.Module, but it would
be required to parse the pretty-printed result.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This new built-in functionality allows callers to find all reachable
paths in a graph based on an array or set of root nodes. See the
updates to policy-reference.md for more details and usage information.
Signed-off-by: Justin Lindh <justin.lindh@webfilings.com>
The function `array.reverse` takes an array as an argument, and returns an array with a reversed order of elements.
The function `strings.reverse` takes a string as an argument, and returns a string with a reversed order of unicode code points.
WASM support is included for both built-ins.
Fixes#3736
Signed-off-by: Kristian Svalland <kristian.svalland@gmail.com>
Add crypto.hmac.* built-in functions for the MD5, SHA-1, SHA-256 and SHA-512 hashing algorithms.
Add documentation for how to contribute new built-in functions.
Fixes: #1740
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Since the golang stdlib function doesn't do any caching, we add the result
to the BuiltinContext.Cache so it's cached, and consistent, within a single
policy evaluation.
There is no decision made here about using netgo or netcgo: we're following
suit wrt how golang expects you to do it: From my understanding, using the
OS means for DNS resolution is the preferred way: it gives you per-host
caching, and it allows the user to affect how DNS resolution works in many
ways.
This means the same logic that applies to all other places where we resolve
domain names into addresses (notably `http.send`) applies to this built-in,
too.
Also:
* workflow/pull_request: don't fail-fast for matrix jobs
Even if one platform fails it would be interesting to see what happens
on the others.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This commit updates the capabilities structure to include the set of
supported future keywords and enhances the parser to accept the
capabilities structure so that callers can restrict what keywords can
be opted into in the first place. This ensures that callers can verify
that policies will parse for a particular version of OPA.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
There are four variants to this: with or without 'some', and with
only one or two lhs arguments:
a) x in xs
b) k, v in xs
c) some x in xs
d) some k, v in xs
(a) and (b) are handled in the parser, and end up in the AST as
calls to `internal.member_2` and `internal.member_3`:
a) internal.member_2(x, xs)
b) internal.member_4(k, v, xs)
The backing builtin functions iterate over their last arguments,
trying to find a match. If they do, they will return `true`.
In all other cases -- no match, or a type in the last argument that
can't be iterated over (not an array, object or set), it will
return `false`.
As such, they can be used with `not` without any restrictions.
(c) and (d) are rewritten in the compiler, where x', v', and k' are
fresh local vars:
c) x' = xs[_]
d) v' = xs[k']
Since `in` is a new keyword, it's enabled gradually: for now, a new
mechanism of future keyword imports is added. There are new option
arguments in the parser methods, and there's a hook in the parser
code updating its set of enabled future keywords whenever it
encounters an import statement like
import future.keywords.in # enables only "in"
import future.keywords # enables all future keywords
Functionally, these are identical right now: there is only one
future keyword, "in".
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
The print built-in function has special-case handling in two ways:
1. Print arguments are wrapped in comprehensions at compile-time to
ensure that undefined values do not short-circuit evaluation. The
evaluator is aware of this rewriting and extracts the actual values
during evaluation.
2. Print arguments cannot contain unsafe/undeclared variables, i.e.,
vars appearing in refs inside of print args _must_ be assigned in
a previous expression in the body. This ensures that print statements
do not affect the semantics of the rule.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
* types: Sort any elements during construction
This changes updates the implementation of the any type to sort
elements during construction. This way the Compare() function does not
have to sort elements before recursing (which can result in data races
if global type instances from the built-in function declarations or
elsewhere are compared.)
Fixes#3793
* capabilities.json: changed ordering
* internal/presentation: fix json error output
Co-authored-by: Torin Sandall <torinsandall@gmail.com>
Co-authored-by: Stephan Renatus <stephan.renatus@gmail.com>
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This will help users for JWT signing using RSA key, because currently
OPA only accepts RSA key in the JWK format.
Fixes: #3765
Signed-off-by: cris-he <cruztiempo@hotmail.com>
* wasm/sdk: check version, call old eval path for ABI 1.1
Fixes#3146.
* docs/wasm: document addition as ABI 1.2
* wasm-sdk: overwrite previous inputs, don't accumulate them
There is a little room for optimization here, should the input
ever grow so large that it eats up too much precious heap space,
we could look into changing this so that the memory used for it
can be reclaimed.
* internal/compiler/wasm: commit generated wasm
I've noticed that since the CI build running on macos-latest doesn't
have docker installed, it cannot update these files itself at build
time. We thus end up with macos binaries that have the wasm binary
data from the main branch, not the PR.
This can be observed from the test failure:
Run make ci-binary-smoke-test-wasm BINARY=opa_darwin_amd64
chmod +x "_release/0.31.0-dev/opa_darwin_amd64"
"_release/0.31.0-dev/opa_darwin_amd64" eval -t "wasm" 'time.now_ns()'
make: *** [ci-binary-smoke-test-wasm] Error 2
{
"errors": [
{
"message": "caller not found: opa_eval (opa_eval)"
}
]
}
Error: Process completed with exit code 2.
Since I had previously commit the CSV data that drives the dead
code elimination process, that optimization had failed to find a
function it expected to have.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This allows users to supply a certificate chain and verify that the leaf
certificate has a path back to the supplied root.
Fixes#3601.
Signed-off-by: James Alseth <james@jalseth.me>
This built-in function makes it possible to get the absolute
difference between to unix timestamps (nanoseconds since epoch)
on the format [year, month, day, hour, minute, second].
Fixes: #3348
Signed-off-by: Andre Håland <andre.haland@gmail.com>
There was a memory leak in the VM that was caused by circular
dependencies on the objects shared between Go and Rust. The circular
dependencies existed because the functions used for the built-ins and
opa_abort were closing over wasmtime.Store-related objects.
By using the Caller to obtain the exports, we can break the circular
dependency and avoid the memory leak.
This commit relies on the fix for
https://github.com/bytecodealliance/wasmtime-go/pull/59 which is in
the development version of wasmtime{-go} that is vendored in a
previous commit.
This change bumps the ABI minor version (1.0 => 1.1)
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
* wasm: emit ABI version as global
This takes inspiration from the proxy-spec (Envoy's Wasm support).
There, it's recorded in an exported function's name. However, it's
been included like that in the spec because it's the least common
denominator among the different languages (potentially) used to
implement proxy-spec. We've got a pretty good grip on our generated
Wasm code, so we do what's noted in proxy-spec as "ideally, we'd do
xyz instead".
However, our ABI version is a simple integer, no semver.
Ref: https://github.com/proxy-wasm/spec/tree/master/abi-versions/vNEXT#proxy_abi_version_x_y_z
* ast.CapabilitiesForThisVersion: include WasmABIVersions
Extending the ast.Capabilities like this is somewhat unsatisfying -- the Wasm ABI has little to do with the ast package. However, moving Capabilities outside of ast in a way that's not introducing import cycles and is backwards-compatible proved to be quite an effort; so let's go with "simple" here.
* capatibilities.json: ensure it is generated with ABI versions
The build tag `generate` is what `go generate` would set, too. We're losing
that in the main.go -> gen-run-go.sh indirection, so we've got to set it
ourselves.
* ci: fix npm-opa-wasm e2e test
The CI build uses a version of OPA built in a previous step -- with the Wasm SDK _disabled_.
To still build Wasm modules, we thus fix the call to use the capabilities.json file from master,
which corresponds to the capabilities of a build of OPA with Wasm SDK enabled.
* docs/content/wasm.md: mention abi version, change headers
There is only one `#` header in a markdown document, so this fixes
that by adding a few `#`. I haven't added it everywhere below
`# Compiling`, but I think the structure is OK now.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This is an implementation of the `json.patch` builtin. Previous discussions
about this include #2839 and #2167.
It does not use an external dependency but rather implements [RFC 6902] directly
on AST terms. This avoids a conversion to JSON as well as the dependency; and
as an added bonus we can make `json.patch` work for sets as well, covering the
full space of AST terms.
In my first implementation I used a mutable approach by first creating a deep
copy and then modifying it in-place. However, this leads to issues with
the cached `hash` values in objects on the path. I replaced this with an
implementation that creates shallow copies. The performance tradeoff is that
smaller patches should be faster; but replacing parts will be slower. Since we
don't know about too many people using this, I think both sides are acceptable.
I vendored the [json-patch-tests] into the test suite in a way that should
make updating them fairly easy. I am also testing the cases disabled there
(since they do work for us!) but I disabled two test cases by adding a new
`opa_disabled` key. These are:
- Us allowing `"foo"` as path (which should be `"/foo"` if you interpret
the RFC strictly).
- A duplicate entry in the JSON patch object which isn't caught by OPA
since it's consistent.
I added some additional tests for sets and things seem to work. I'm going
to try out this new functionality in our larger codebase to see if any issues
come up, but I expect it to hold up. Update: we've been using this builtin
and haven't seen any issues so far.
[RFC 6902]: https://tools.ietf.org/html/rfc6902#section-4.4
[json-patch-tests]: https://github.com/json-patch/json-patch-tests
Signed-off-by: Jasper Van der Jeugt <jasper@fugue.co>
The builtin hash algorithms hex encode the result. To use the hash functions
there must be possible to decode the value and re-encode it in the expected format.
This enables validation of x5t/x5t_s256, which are base64 url encoded hashes.
Fixes: #2849
Signed-off-by: Johannes Larsson <johannes.a.larsson@gmail.com>
This commit adds a new builtin to merge adjacent subnets and return the
smallest possible list of CIDRs.
To help with computing CIDR blocks between two
IP networks, an implemetation from https://github.com/cilium/cilium
is leveraged.
Fixes: #2692
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
This builtin is the reverse of the encode_object builtin and
makes it easier to use the URI query parameters in policies
Fixes#2647
Signed-off-by: Frederic <frederic.vanreet@icloud.com>
Emil Volckmar Ry