Commit Graph

114 Commits

Author SHA1 Message Date
Stephan Renatus 04a3523b22 ast+format: introduce new keywords for rule heads: if and contains
`contains` provides an alternative way to declare partial sets:

    p contains x {
      x := { "foo": "bar"
    }

which is the same as

    p[x] {
      x := { "foo": "bar"
    }

The keyword is enabled by importing `future.keywords.contains`, and
when it _is enabled_, the format will be used for all partial sets in
that file for pretty-printing.

`if` is a new keyword allowing for more readable rule definitions:

The syntax is

    NAME [if] { EXPR [EXPR...] }

and the is a shorthand allows dropping the braces around the expression
if there is only one:

    NAME if EXPR

For example, this allows expressions like

    allow if not deny
    f(xs) if every x in xs { x != "foo" }

The one exception here are partial sets: they cannot use `if` UNLESS
they use `contains`:

    p[x] { x := "foo" }            # valid
    p contains x { x := "bar" }    # valid
    p contains x if { x := "bar" } # valid
    p[x] if { x := "foo" }         # invalid

This is because we want to interpret that differently (as an object
rule defining `p.foo = true`) in the near future.

The formatter works in the same way: if `future.keywords.if` is imported, it
will be used where it can be used.

We don't want to be too eager when it comes to introducing syntactic sugar.

So this will be rewritten, because head and body expression are on the same
line:

    p := 5 if { time.day_of_week() == "Monday" }

    # => p := 5 if time.day_of_week() == "Monday"

but this won't:

    p := 5 if {
       time.day_of_week() == "Monday"
    }

The rationale here is that if the policy author decided that they want this on
an extra line, we won't mess with it.

This also sidesteps the need to check if both the head and the single body
expression have a comment.

This change includes various docs updates. Notable exceptions are the GK docs,
since it will take a while for these keywords to be come available there; and
the frontpage: merging a PR would update the frontpage immediately, and we
don't want to show something there that isn't available in the latest release.

Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
2022-06-22 10:36:06 +02:00
Philip Conrad 1fde1ad235 built-ins: Add new GraphQL builtins.
This commit vendors in github.com/vektah/gqlparser, and provides the
implementation for the following graphql built-ins:

 - graphql.parse
 - graphql.parse_and_verify
 - graphql.parse_query
 - graphql.parse_schema
 - graphql.is_valid

The test suite is comprised of the classic "Star Wars" examples from the
official GraphQL docs, along with a host of syntax examples ported over
from the underlying GraphQL parser's test suite.

AST objects returned by the parse_x APIs are currently pruned for brevity.

Fixes #4283

Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
2022-05-27 10:26:42 -07:00
Anders Eknert af5141bfa0 Update docs to prefer := over = (#4706)
Seems like we missed this in a "few" places :)

Signed-off-by: Anders Eknert <anders@eknert.com>
2022-05-24 23:10:11 +02:00
Stephan Renatus e8deba62bd types, ast, docs: Add support for named function arg declarations
This commit adds support for named argument declarations for built-in
functions as well as additional metadata/annotations on built-in
functions (e.g., descriptions, categories, etc.) This commit allows us
to generate a data file (builtin_metadata.json) that other tools can
consume to improve the Rego authoring experience.

Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>

Co-authored-by: Torin Sandall <torinsandall@gmail.com>
2022-05-24 09:38:07 -07:00
MM Zeeman 4cb0c7b7fe docs/policy-refernece: Fix import rule in grammar (#4690)
`Package` should be `ref`.

Fixes #4689.

Signed-off-by: Maas-Maarten Zeeman <mmzeeman@xs4all.nl>
2022-05-15 21:03:34 +02:00
Philip Conrad 53cb5f40f9 topdown: Add units.parse builtin (#4676)
This function works on all base decimal and binary SI units of the set:

    m, K/Ki, M/Mi, G/Gi, T/Ti, P/Pi, and E/Ei

Note: Unlike `units.parse_bytes`, this function is case sensitive.

Fixes open-policy-agent#1802.

Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
2022-05-11 16:54:51 +02:00
Anders Eknert f8151c47ed Fix incorrect padding claims (#4657)
Signed-off-by: Anders Eknert <anders@eknert.com>
2022-05-04 08:14:45 +02:00
Johan Fylling c023bb6ebe docs: Adding example for rego.metadata.role() usage (#4640)
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
2022-04-28 15:30:55 +02:00
Stephan Renatus b481f000c7 topdown/net: require prefix length for IPv6 in net.cidr_merge (#4613)
There are no default prefixes in IPv6, so if an IPv6 without a prefix is fed into
net.cidr_merge, we'll return a non-halt error now.

Before, we'd fail in various ways if a prefix-less IPv6 was fed into
`net.cidr_merge`. With only one, we'd return `[ "<nil>" ]`, with two,
we'd panic.

Fixes #4596.

Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
2022-04-26 19:03:24 +02:00
Torin Sandall 54827f3ef4 docs/content: Update docs to use := at the file-level
This partially addresses #4599.

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2022-04-20 09:27:39 -07:00
Johan Fylling aeda97e8ee ast: Extending support for file-level assignments (:=) (#4583)
Updated support for:
* default values
* rules with `else` keyword
* partial rules
* functions

Fixes: #4555

Signed-off-by: Johan Fylling <johan.dev@fylling.se>
2022-04-20 13:28:51 +02:00
Johan Fylling c6226621c4 ast: Adding rego.metadata.* built-in functions (#4537)
New functions:
* rego.metadata.chain(): returns the chain of metadata, starting from the active rule, going outward
* rego.metadata.rule(): returns the metadata for the active rule

Signed-off-by: Johan Fylling <johan.dev@fylling.se>
2022-04-13 11:46:11 +02:00
Stephan Renatus 76706723ae topdown/http: parse (and cache) yaml response (#4535)
This is similar to what we currently do for application/json responses, or when
force_json_decode is set to true: We parse the result, and according to how it's
configured, cache either the serialised or deserialised content.

Now, we'll also take care of yaml-related content-types, application/yaml and
application/x-yaml; and enable forcing yaml-decoding via force_yaml_decode.

Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
2022-04-05 09:59:08 +02:00
Stephan Renatus b005ba33f5 policy-reference: update EBNF to include "every" and "some x in ..."
Fixes #4216.

Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
2022-02-21 09:53:50 +01:00
Charlie Egan 301efc6997 Alter object.get to support nested key array
This commit extends the go and wasm implementations of object.get to
allow a key to also be an array.

When passed an array, each element in the array will be used as a key in
turn. This allows values at deeply nested paths to be extracted from
objects.

It also supports getting indexes of nested arrays.

The functionality was originally inspired by Ruby's Hash.dig function:
https://ruby-doc.org/core-2.3.0_preview1/Hash.html#method-i-dig however
we opted to include the behavior in object.get instead after being
uncertain 'dig' was a commonly understood name.

Signed-off-by: Charlie Egan <charlieegan3@users.noreply.github.com>
2022-01-27 22:01:01 +01:00
Anders Eknert caffcec07b builtins: add object.union_n
Fixes: #4012

Signed-off-by: Anders Eknert <anders@eknert.com>
2022-01-23 20:24:57 +01:00
Justin Lindh 258501134d built-ins: add graph.reachable_paths (#4205)
This new built-in functionality allows callers to find all reachable
paths in a graph based on an array or set of root nodes. See the
updates to policy-reference.md for more details and usage information.

Signed-off-by: Justin Lindh <justin.lindh@webfilings.com>
2022-01-17 09:31:31 +01:00
Kristian Svalland 3250a2c858 wasm: Add native support for json.is_valid (#4204)
wasm: Add support for WASM and simple tests.
internal: Add opa_json_is_valid to map of wasm built-ins.
docs: Indicate that WASM support is now available for json.is_valid.

Fixes #4140

Signed-off-by: Kristian Svalland <kristian.svalland@gmail.com>
2022-01-11 07:47:33 +01:00
Shuhei Kitagawa 52ddfd9149 topdown: Support indexof_n built-in function (#4172)
Fixes https://github.com/open-policy-agent/opa/issues/4155

Signed-off-by: shuheiktgw <s-kitagawa@mercari.com>
2022-01-06 09:58:27 +01:00
Kristian Svalland 6f81c4a620 Add array.reverse(array) and strings.reverse(string) built-in functions. (#4161)
The function `array.reverse` takes an array as an argument, and returns an array with a reversed order of elements.
The function `strings.reverse` takes a string as an argument, and returns a string with a reversed order of unicode code points.
WASM support is included for both built-ins.

Fixes #3736

Signed-off-by: Kristian Svalland <kristian.svalland@gmail.com>
2021-12-27 12:47:39 +01:00
Johan Fylling edf5f25951 topdown: Fixing nanos int overflow issue for time.* built-in functions (#4117)
* topdown: Fixing nanos int overflow issue for time.* built-in functions

The go Time.UnixNano() function result is undefined for dates that
cannot fit into an int64 when converted to Unix time in nanoseconds.

Updating built-in functions to return error if date is too low/high.

Fixes: #4098
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
2021-12-10 12:58:11 +01:00
Johan Fylling 76547e55d1 topdown: Add built-in HMAC functions (#4100)
Add crypto.hmac.* built-in functions for the MD5, SHA-1, SHA-256 and SHA-512 hashing algorithms.

Add documentation for how to contribute new built-in functions.

Fixes: #1740
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
2021-12-09 13:48:17 +01:00
Amit Levy 2b25dc06c8 docs/content: Make spacing consistent (#4057)
Signed-off-by: Amit Levy <amitlevy347@gmail.com>
2021-11-28 17:57:14 +01:00
Torin Sandall ec71ead8d9 docs: Update description of urlquery functions
Now that we support urlquery.decode_object the associated issue can
closed with this fix of the description (i.e., don't use the term
"serialize" in this context.)

Fixes #1592

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2021-11-24 15:35:36 -08:00
Stephan Renatus 8a1aab376f ast+topdown: add net.lookup_ip_addr built-in function (#3995)
Since the golang stdlib function doesn't do any caching, we add the result
to the BuiltinContext.Cache so it's cached, and consistent, within a single
policy evaluation.

There is no decision made here about using netgo or netcgo: we're following
suit wrt how golang expects you to do it: From my understanding, using the
OS means for DNS resolution is the preferred way: it gives you per-host
caching, and it allows the user to affect how DNS resolution works in many
ways.

This means the same logic that applies to all other places where we resolve
domain names into addresses (notably `http.send`) applies to this built-in,
too.

Also:

* workflow/pull_request: don't fail-fast for matrix jobs

Even if one platform fails it would be interesting to see what happens
on the others.

Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
2021-11-16 07:39:14 +01:00
Stephan Renatus 94ec32e402 website: bump to hugo to 0.88.1, adapt code and live-blocks (#3986)
* website: bump hugo, updated templates

For some reason, the "and" trick to step around undefined values no longer
worked. So now, we're wrapping {{ if }} conditionals instead: the outer one
checks that the value is defined, the inner one asserts something on its
value.

* live-blocks/preprocess: add wrapper div, remove newline

Somewhere between 0.55 and the most recent version, Hugo stopped emitting
that div on purpose for languages it didn't know.

Since we depend on it for further processing, we're adding it back.

Also, a trailing newline for code elements was introduced, and that broke
our codemirrors integration -- every code entry would be folllowed by an
empty line. Now, we're removing that in the same preprocess step.

* website: fix info|danger blocks
* website/config: trust markdown to include html

Fixes #3787.

Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
2021-11-09 09:45:31 +01:00
Stephan Renatus d83d7de264 docs/website: add warning to 'http.send'+extension docs, introduce danger shortcode (#3922)
Fixes #3893.

Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
2021-10-26 12:00:58 +02:00
Stephan Renatus 527dc639dc docs/policy-reference: fix and clarify 'in' example (#3917)
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
2021-10-23 07:39:00 +02:00
Stephan Renatus bbd1084eb6 docs/policy-reference: add 'in' operator examples (#3914)
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
2021-10-22 20:01:18 +02:00
Torin Sandall 64522f92dd docs: Update debugging section for print calls
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2021-10-14 09:31:16 -07:00
Cris He 22d505fd9e built-ins: decode a RSA private key into JWK format (#3783)
This will help users for JWT signing using RSA key, because currently
OPA only accepts RSA key in the JWK format.

Fixes: #3765

Signed-off-by: cris-he <cruztiempo@hotmail.com>
2021-09-17 07:03:16 +02:00
Kirk Patton d04707075a add socket support for http.send (#3667)
Fixes #3661.

Signed-off-by: Kirk Patton <kpatton@verizonmedia.com>
2021-08-17 21:39:16 +02:00
Stephan Renatus 349c7a0333 docs: type_name builtin is natively implemented in Wasm (#3671)
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
2021-07-27 07:28:31 +02:00
Rina Ueno 320f9a6b06 docs: add the description about Note event for debugging (#3659)
Signed-off-by: Rina Ueno <rina.ueno.yt@hitachi.com>
2021-07-26 14:42:04 +02:00
Ashutosh Narkar 4f8c8fd215 topdown: Add caching mode parameter to http.send
This commit adds a new parameter to http.send to
control how items are added to the inter-query cache.
Currently two modes are supported which allow users
to decide if they prefer cache memory conservation or low
latency during cache lookups.

Fixes #3599

Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
2021-07-15 10:49:17 -07:00
Torin Sandall 44436dc131 topdown: Improve rand.intn implementation (#3646)
This commit combines a few improvements to the rand.intn
implementation:

* Zero and negative integers do not cause an error. In most cases, we
  try to make built-in functions return sensible results (rather than
  halting evaluation or being undefined.) In this case, it's easy
  enough to handle zero and negative integers.

* Use the built-in context seed to generate the source for the rand
  call. This helps ensure that rand.intn calls can be reproduced and
  ensures that (by default) numbers genreated by rand.intn differ
  across runs.

* Use dedicated cache key type instead of overloading uuidCachingKey.

* Add test coverage for caching, partial eval, etc.

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2021-07-14 08:06:20 +02:00
Stephan Renatus 6b0f5787f2 docs/policy-reference: add missing ` (#3617)
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
2021-07-06 17:22:45 +02:00
James Alseth c354e3c4f2 Add crypto.x509.parse_and_verify_certificates (#3602)
This allows users to supply a certificate chain and verify that the leaf
certificate has a path back to the supplied root.

Fixes #3601.

Signed-off-by: James Alseth <james@jalseth.me>
2021-07-05 21:59:59 +02:00
Stephan Renatus 752e309725 http.send: ignore tls_use_system_certs on windows (#3613)
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
2021-07-02 15:12:05 +02:00
Pat Myron 47feea7cbb docs: syntax highlighting EBNF grammar (#3588)
https://docs.github.com/en/github/writing-on-github/working-with-advanced-formatting/creating-and-highlighting-code-blocks#syntax-highlighting

Signed-off-by: Pat Myron <pmyron@nd.edu>
2021-06-27 08:45:25 +02:00
Olamide Omolola c9c7bac009 topdown: change the tls_use_system_certs default to true
This commit changes the default of `tls_use_system_certs`
parameter for `http.send` from `false` to `true`

Fixes #2271

Signed-off-by: Olamide Omolola <omololaolamidex@gmail.com>

Fixup to commit 5c213e5

Signed-off-by: Olamide Omolola <omololaolamidex@gmail.com>

Fixed variables name changes as suggested by @anderseknert

Signed-off-by: Olamide Omolola <omololaolamidex@gmail.com>

Amended test as suggested by @srenatus

Signed-off-by: Olamide Omolola <omololaolamidex@gmail.com>
2021-05-22 09:02:09 +02:00
André Håland 458d87489b built-ins: time.diff function (#3358)
This built-in function makes it possible to get the absolute
difference between to unix timestamps (nanoseconds since epoch)
on the format [year, month, day, hour, minute, second].

Fixes: #3348

Signed-off-by: Andre Håland <andre.haland@gmail.com>
2021-04-13 14:30:08 +02:00
Jon Zeolla ce2aeaf163 Fix policy reference documentation typo
Signed-off-by: Jon Zeolla <zeolla@gmail.com>
2021-03-22 17:11:18 -07:00
Anders Eknert 80de059208 Remove any() and all() built-ins from policy reference
Since there are better ways of doing what they do and they tend to confuse
people new to Rego we'll hide these from the policy reference for now. Will
eventually be deprecated and removed with issue #2437.

Signed-off-by: Anders Eknert <anders@eknert.com>
2021-03-09 13:00:37 -05:00
Ashutosh Narkar e9cc8551f9 wasm: Add native support for json.filter builtin function
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
2021-01-19 11:08:32 -08:00
Ashutosh Narkar ba5cda16ee wasm: Add native support for json.remove builtin function
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
2021-01-15 09:26:27 -08:00
Anders Eknert fa94a2d466 Add ceil and floor built-ins
Closes #2930

Signed-off-by: Anders Eknert <anders@eknert.com>
2021-01-07 16:22:41 -05:00
Ashutosh Narkar f7d8b1ca9b wasm: Add native support for object.remove and object.union builtin function
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
2021-01-05 16:48:43 -08:00
Anders Eknert ce8dfb9669 docs: Add 'some' to reserved names
Signed-off-by: Anders Eknert <anders@eknert.com>
2020-12-16 19:37:46 -05:00
Ashutosh Narkar 1a708a9c03 wasm: Add native support for graph.reachable builtin function
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
2020-12-15 09:37:50 -08:00