Commit Graph

768 Commits

Author SHA1 Message Date
Patrick East f84f4674a1 Add support for evaluating Wasm bundles
This is largely plumbing changes required to get Wasm modules loaded
from bundles and configured as external resolvers for evaluations.

Signed-off-by: Patrick East <east.patrick@gmail.com>
2020-11-06 15:12:38 -05:00
Torin Sandall 2c2fe06ca3 docs: Add section on entrypoints in wasm modules
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-11-02 13:45:02 -05:00
Torin Sandall f622582e88 wasm: Fix serialization of non-string object keys
Previously the C library was not escaping non-string object
before serializing them--this would break callers that expect
valid JSON output from opa_json_dump. With this change,
opa_json_dump will serialize/escape non-string object
keys. opa_value_dump retains the old behaviour.

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-10-30 16:54:19 -07:00
Jasper Van der Jeugt d8947db27d topdown: add yaml.is_valid and json.is_valid
Signed-off-by: Jasper Van der Jeugt <jasper@fugue.co>
2020-10-27 10:30:15 -04:00
Torin Sandall 5fcb3f0451 wasm: Add support for parsing and dumping set literals
This commit adds support into the C library for parsing and dumping
set literals without representing them as arrays (we use the Rego
convention of curly-braces and set() for emptiness.) The parsing and
dumping that supports set literals is exposed via new opa_value_parse
and opa_value_dump functions (respectively).

Fixes #2773

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-10-26 14:57:58 -04:00
Patrick East 539543ed01 wasm: Add data patching API's to wasm helper bin
There are three new API's implemented and exposed from the C code:

```
opa_value_add_path
opa_value_remove_path
```

and a new helper:

```
opa_object_remove
```

The first two provide similar functionality as the OPA stores "add"
and "remove" op (same style of path). The main difference for the add
is that it will create intermediate objects as required, we do not
have a separate `mkdir` operation like the store does.

Signed-off-by: Patrick East <east.patrick@gmail.com>
2020-10-23 12:26:51 -07:00
Richard Simpson ab9e9add70 plugin/rest: Adds AWS Web Identity support
Adds support for signing S3 requests using Web Identity credentials. Specifically supports the Env Var version that's used in EKS IRSA.

Closes: #2463
Signed-off-by: Richard Simpson <richardsimpson@outlook.com>
2020-10-23 10:01:31 -07:00
Torin Sandall 544c2b0cbd plugins/bundle: Add file size limit configuration option
This commit lets users override the 1GB file size limit on the bundle
reader with a configuration setting.

Fixes #2781

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-10-21 11:00:24 -04:00
Kanishk Kakar 75f98c0f50 docs/content: Fix typo in Introduction page.
Change statement from "language that let’s you specify policy" to
"language that lets you specify policy."

Signed-off-by: Kanishk Kakar <kanishk.kakar@gmail.com>
2020-10-19 09:53:28 -07:00
Nikos Silvestros 2daeeccaaa docs: Update Kubernetes tutorial to create SAN certificate
Create SAN certificate for OPA service because Go 1.15 deprecated CN support.

Signed-off-by: Nikos Silvestros <nsilvestros@gmail.com>
2020-10-19 09:52:13 -07:00
Anders Eknert a7aed3aab2 Add REST client option for OAuth2 client credentials
Allows OPA to obtain OAuth2 access tokens and present them as bearer tokens for authentication at remote endpoints.

Fixes #1205

Signed-off-by: Anders Eknert <anders.eknert@bisnode.com>
2020-10-15 12:43:44 -07:00
Sebastian Cevallos 50defaea9a Fix incorrect anchor link to Grammar
Signed-off-by: scevallos <sebastianlcevallos@gmail.com>
2020-10-15 12:32:06 -07:00
Torin Sandall cc4fc16624 docs: Add note about valid package syntax
Fixes #2778

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-10-13 13:29:03 -04:00
Ashutosh Narkar 63560e0d1e topdown/http: handle http.send errors in-band
This change updates how errors from http.send are handled.
By default, an error returned by `http.send` halts the policy evaluation.
This commit allows users to return errors in the response object
returned by `http.send` instead of halting evaluation.

Fixes: #2187

Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
2020-10-09 09:11:19 -07:00
Torin Sandall ebf657ca53 docs: Mention that /metrics and /health are exposed on normal listener
Fixes #2727

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-10-06 09:21:12 -04:00
Ashutosh Narkar f8fcc2c9e0 topdown/http: Add support for controlling caching in http.send
This commit adds two new fields to the http.send builtin
that allow the user to override the caching directives defined
by the server and thus get more control over the caching
behavior.

Fixes: #2666

Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
2020-10-05 17:40:47 -07:00
Torin Sandall c560ec3bac plugins/bundle: Tweak bundle save directory
Store bundles under a specific sub-directory of the new working
directory (as the same directory may be useful in future for other
persitence features.) Also, update the docs to mention where the files
are saved to as this was unclear from the user's perspective.

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-10-05 15:59:26 -07:00
Stephan Renatus aa95762c44 golang: 1.13.7 -> 1.14.9
With 1.13.x no longer supported, we should pick the latest 1.14 minor release.

Signed-off-by: Stephan Renatus <srenatus@chef.io>
2020-10-05 15:58:41 -07:00
Ashutosh Narkar 5a79a45e4f plugins/bundle: Support for saving and reading bundles from disk
This commit adds support to persist and load bundles from disk.
A new field is introduced in OPA's bundle configuration that can
be optionally set to enable OPA to write and read bundles from disk.
This feature will allow OPA to serve policy decisions in scenarios
such as OPA being unable to communicate with the bundle server.

Fixes #2097

Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
2020-09-30 15:17:51 -07:00
Anders Eknert 24b4bf2f77 Add link to new IDEA plugin
Signed-off-by: Anders Eknert <anders.eknert@bisnode.com>
2020-09-28 10:42:36 -07:00
Ashutosh Narkar e9aee648b9 topdown: Add net.cidr_merge function to produce smallest possible list of CIDRs
This commit adds a new builtin to merge adjacent subnets and return the
smallest possible list of CIDRs.

To help with computing CIDR blocks between two
IP networks, an implemetation from https://github.com/cilium/cilium
is leveraged.

Fixes: #2692

Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
2020-09-24 18:03:23 -07:00
Peter Sullivan 7f0399b3f2 docs/content: Update Envoy Authorization Tutorial
Updated envoy version and links. Updated service type and minikube commands to tunnel to the service for consistency across minikube versions and drivers

Signed-off-by: Peter Sullivan <pvsone@gmail.com>
2020-09-21 12:25:54 -07:00
Ashutosh Narkar 8cb34e48c2 topdown: Address negative duration for the current age of http response
The current age of a http response is calculated as the difference
between the current time and the value contained in the "Date" response
header. There are couple of scenarios that could lead to the current age
being represented as a negative duration.

1. Since the value of "Date" response header is parsed using Go's
time.Parse method, it does not contain a monotonic clock reading. As a result,
the time.Sub method uses wall clock readings to determine the difference between
current time and the parsed version of the response time.

2. The server could set a value for the "Date" response header which may not be a true indication of
when the response was generated.

This change updates the logic that determines whether a cached response is fresh or not,
to treat the resposne as stale if the current response age is represented as a negative duration.

Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
2020-09-21 10:46:53 -04:00
AlexsJones 59a7e430ec chore(spelling): fix spelling in extensions.md and rest-api.md docs
Signed-off-by: AlexsJones <alexsimonjones@gmail.com>
2020-09-15 18:00:48 -04:00
rtfee cba68b1dda updated the Scalr integration title
Signed-off-by: rtfee <ryan.fee625@gmail.com>
2020-09-15 10:53:14 -04:00
dependabot[bot] 724c0bb0be build(deps): bump node-fetch in /docs/website/scripts/live-blocks
Bumps [node-fetch](https://github.com/bitinn/node-fetch) from 2.6.0 to 2.6.1.
- [Release notes](https://github.com/bitinn/node-fetch/releases)
- [Changelog](https://github.com/node-fetch/node-fetch/blob/master/docs/CHANGELOG.md)
- [Commits](https://github.com/bitinn/node-fetch/compare/v2.6.0...v2.6.1)

Signed-off-by: dependabot[bot] <support@github.com>
2020-09-14 10:31:14 -07:00
scevallos 56afac8817 Make correction to json.remove syntax in example
Signed-off-by: scevallos <sebastianlcevallos@gmail.com>
2020-09-09 13:25:07 -07:00
rtfee 7827766b68 added Scalr integration
Signed-off-by: rtfee <ryan.fee625@gmail.com>
2020-09-09 14:12:39 -04:00
Anders Eknert 03187af692 Add pre-commit integration
Signed-off-by: Anders Eknert <anders@eknert.com>
2020-09-02 09:32:03 -07:00
Torin Sandall 958a8ba835 docs: Add note about logs.EventV1 fields in extensions doc
Fixes #2664

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-09-01 11:09:24 -04:00
Rob Schoening 2f2bb7f0da spelling error
Signed-off-by: Rob Schoening <rob@soluble.ai>
2020-09-01 10:40:51 -04:00
Frederic 870f28ff4a topdown: Add urlquery.decode_object builtin
This builtin is the reverse of the encode_object builtin and
makes it easier to use the URI query parameters in policies

Fixes #2647

Signed-off-by: Frederic <frederic.vanreet@icloud.com>
2020-08-31 11:55:15 -04:00
Torin Sandall dd78d44524 capabilities: Add capabilities.json files for patch releases
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-08-26 10:03:52 -04:00
Torin Sandall b5f4c57638 test: Move test case loader utility into test package
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-08-20 12:48:20 -04:00
Torin Sandall de04593f4c docs: Add example test case implementation
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-08-20 12:48:20 -04:00
Ashutosh Narkar 30271bc18b plugins/discovery: Set the last request and last successful request in discovery status
This commmit sets the values for the last request and last successful request
in the discovery bundle status.

Fixes: #2630

Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
2020-08-19 05:10:05 -07:00
Ashutosh Narkar b48aba82b0 plugins/rest: Add response header timeout for REST client
This commit adds a new configurable timeout to the Services
config to set the amount of time to wait for the server's
response headers. With this change, the client will no longer
wait indefinitely for the HTTP request to complete.

Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
2020-08-19 05:10:05 -07:00
Torin Sandall 17972357e5 docs: Update release steps to use 'git add' to include capabilities
The release patch will include a new file in the repo so we need to
use git add to include that file.

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-08-18 13:51:38 -04:00
Torin Sandall fda63bd26a compile: Add support for unknowns other than input
Previously the build command and the compile package would only mark
input as unknown. If documents under data needed to be treated as
unknown, there was no solution. This commit updates the compile
package to infer unknowns based on the bundle roots. If the policy
refers to a data document _outside_ of one of the bundle roots, that
data document will be marked as unknown during optimization/partial
eval.

Fixes #2581

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-08-17 10:34:38 -04:00
Torin Sandall 0d54ba204f docs: Update external data page description of sync-pull option
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-08-14 14:00:46 -04:00
Torin Sandall e8806c15eb docs: Remove openable tag from test examples
The test examples cannot be opened into the playground because they
require multiple files and the playground UI does not support that
today.

Fix #2073

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-08-12 15:53:59 -04:00
Patrick East 19f1d89d03 docs: Update live-blocks dependencies
Using `npm audit fix` update dependencies of the live-blocks tooling.

This corrects all severe ones that can be corrected without pulling
library versions that include breaking changes.

Signed-off-by: Patrick East <east.patrick@gmail.com>
2020-08-12 12:37:21 -07:00
dependabot[bot] 2105a28fbf build(deps-dev): bump serialize-javascript
Bumps [serialize-javascript](https://github.com/yahoo/serialize-javascript) from 2.1.2 to 3.1.0.
- [Release notes](https://github.com/yahoo/serialize-javascript/releases)
- [Commits](https://github.com/yahoo/serialize-javascript/compare/v2.1.2...v3.1.0)

Signed-off-by: dependabot[bot] <support@github.com>
2020-08-12 10:05:30 -07:00
Ashutosh Narkar bd5009c994 docs: Add bundle for opa-envoy policies
This change adds a sample bundle which can be exercised as
part of the quick start instructions for opa-envoy plugin.

This commit also updates the shortcode to use the latest-envoy tag
for the opa-envoy image for latest and edge versions of the OPA docs.

Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
2020-08-10 10:41:36 -07:00
Torin Sandall 7821831dfa server: Deprecate the watch and partial query parameters
Related: https://github.com/open-policy-agent/opa/issues/2265
Related: https://github.com/open-policy-agent/opa/issues/2266

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-08-10 10:24:50 -04:00
Teemu Koponen b5b3280d22 bundle: Use the standard key id in the JWT header.
This deprecates the key id claim. The claim will be injected in
signing for backwards compatibility and checked in verifying, if the
header has no kid.

Signed-off-by: Teemu Koponen <koponen@styra.com>
2020-08-06 20:58:11 -04:00
Ashutosh Narkar 7f65b04561 Add a new inter-query cache to cache responses across queries
This commit adds a new inter-query cache that built-in
functions can use to cache responses across queries.

The OPA config includes a new "caching" field that can be used
to set the size of the cache. By default there is no limit.

This change also updates `http.send` to optionally utilize the
inter-query cache.

Fixes #1753

Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
2020-08-06 14:33:18 -07:00
Torin Sandall ad4b7b3a57 wasm: Remove references to the heap top address
SDKs don't need to set the heap top address because it's managed by
the malloc implementation and should be left intact across
executions (otherwise the malloc implementation will keep allocating
more memory from the host because the top has been reset across
executions.)

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-08-04 12:57:08 -04:00
Torin Sandall 8b550ce67b ast: Marshal non-string object keys when converting to interface{}
Previously, when OPA attempted to convert an ast.Object to interface{}
it would error if the ast.Object contained any keys that were not
ast.String values. This behaviour was implemented because JSON only
supports object keys as strings.

This commit changes the conversion implementation to simply JSON
marshal non-string object keys when they are encountered. This way
we avoid runtime errors which can be difficult to debug.

Fixes #516

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-07-27 11:04:05 -04:00
Patrick East 35b78b8afb CI: Skip asset upload/push when missing config
Previously we had some required configuration for the Github Actions,
and forks of OPA would need to set them _and_ have the underlying
infra configured (eg, docker registries, s3 bucket, etc).

Now it will check if the secrets are set, and if any required ones
are missing it will skip the steps.

This significantly lowers the bar for OPA forks to be able to run the
normal action workflows without getting errors. The only lost
functionality is primarily around publishing release assets, which
is not required for dev forks, and other forks can opt int to pieces
they care about (eg, only want to publish docker images and no
s3 release assets).

Signed-off-by: Patrick East <east.patrick@gmail.com>
2020-07-24 10:49:19 -04:00