Prometheus metrics can give much of insight into OPA's health.
Run-time metrics are a natural part of the application state
so having them in status update seems like a right change
that can help server understand what's going in with the OPA
instance.
The commit also encapsulates all prometheus-related code in one package
and abstracts it with generic interface so that it would be possible
to add other metrics providers
Addresses #1606
Signed-off-by: Stan Lagun <stan@styra.com>
These changes update the travis build to produce windows/darwin
binaries in addition to the linux binary and push them to the S3
bucket holding the edge binaries. This lets users on windows/darwin
easily obtain the binary for tip of master. It also enables the live
docs build which is coming soon.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
The goal is to have the Kubernetes admission control content organized
into one section. This should make it easier to incorporate Gatekeeper
content in the future.
Since the layouts are not versioned the layout logic had to be
enhanced to filter out sections that have no content for the current
version. Doing a semantic version comparison was considered but it was
unclear how to implement that properly within the Hugo template
system. Since it's unlikely that we will want to ever display empty
sections this approach seems fine.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
The release-builder image is no longer required because we only need
the base Go image to build the OPA binaries. The nodejs dependency was
only required for the docs which are handled by netlify now.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
The documentation was updated part way through adding the Prepare
helpers on the Rego object. Later on when PrepareForEval and
PrepareForPartial helpers were added and we stopped re-using
PartialResult this was left behind and is now incorrect.
Fixes: #1619
Signed-off-by: Patrick East <east.patrick@gmail.com>
This is not complete (e.g., more detail on management APIs would be
great, detail on CLI invocation would be nice, detail on partial
evaluation would be very helpful, etc.) but it provides a starting
point.
Fixes#372
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
While adding support for the some keyword in the REPL and fixing #1104
there was a regression where statements like `input = 1` would always
be interpreted as rules. We made a decision a long time ago that the
first time an expression like `input = 1` was encountered that a rule
would be declared but that subsequent similar expressions (e.g., input
= 1 or input = {"foo":"bar"} or ...) would perform a comparison. The
regression broke this for cases where the left hand side was a
reference to a global document (i.e., input or data). This commit just
fixes the regression by updating the global check to account for refs.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This avoids duplicating any logic that may be required to construct
rules from := expressions. Currently the only extra bit of logic is to
set the assignment flag on the rule head. This change lets us
determine whether the rules are unset in the REPL in a more
declarative manner (i.e., if it's an assignment rule then it will
unset in all cases) and ensure that the assignment operator is not
lost in the show command output.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
These changes update the AST to allow users to declare rules with the
:= operator. The compiler checks that the rule has not been declared
elsewhere in the package. These changes help bridge the gap between
the REPL and modules.
Fixes#1541
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
As reported in #1582, there are some issues with parsing variables that
start with `false`, `true` or `null`.
Two possible fixes for this issue are:
1. Reordering the `Term` rule:
Term <- Comprehension / Composite / Scalar / Call / Ref / Var
2. Fixing the bug in `Scalar`: e.g. `nulls` should _not_ parse as a
scalar by ensuring these keywords are not prefixes of a variable.
As far as I know this is a fairly common technique in grammars.
It's bikeshedding the issue a bit but I would argue that (1) is not a
great solution: it really just works around the bug by moving things
around in `Term`: the bug may pop up again when reusing `Scalar` in
other places.
In order to do (2) nicely I also refactored the parser for variables
into one for var's first and subsequent characters.
Signed-off-by: Jasper Van der Jeugt <jasper@fugue.co>
This commit removes the deprecated diagnostic feature from the
server. The feature has been deprecated since November 2018 and it was
essentially unused at the time so it should be safe to
remove. Removing the diagnostic support from the server saves having
to perform an extra policy evaluation in the server.
Once the buffer is removed from the runtime.Params struct the related
issue can be closed (there is still one known user of that so it has
been left intact for backwards compatibility.)
Ref #1052
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This commit adds `http_request_cancellations` counter to the Prometheus
metrics. Having such a counter helps identify requests that were timed
out or the client just got disconnected before the request result was
computed.
Signed-off-by: Stan Lagun <stan@styra.com>
This commit adds two checks to the in-memory store to detect improper
use of transactions:
1. Mark aborted/committed transactions as stale and error if an
operation is attempted on a stale transaction. Previously callers
could perform unsafe concurrent operations on stale transactions
without noticiing.
2. Check that supplied transactions are from the underlying store and
not another store. We could consider deprecating the store APIs and
moving them onto the transaction to prevent this kind of mistake in
the future.
With these changes the store will panic on unregister and abort
calls if any of these invariants are violated. Panicing is preferable
to failing silently.
Note, we still have the issue of recursive transactions resulting in
deadlock. Our options there are to implement more sophisticated
locking inside the in-memory store or modify the API so that the store
can update the passed context.Context with a sentinel value.
Fixes#1594
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This change brings in support for multiple bundles to be downloaded
and activated OPA.
This is enabled by using the new config option `bundles` to define
the bundles, and deprecates the older `bundle` option.
The new `bundles` keyword and structure is propagated through to the
decision logs, status API, provenance, stored manifests, etc. Check
out the doc changes for all the updated structures.
That being said any existing configuration using `bundle` will *not*
see the new structure, everything is intended to be backwards
compatible (almost to a fault).
Fixes: #721
Signed-off-by: Patrick East <east.patrick@gmail.com>
This will allow others consuming OPA as a library to have easier
access to manage/query the bundle manifests.
Signed-off-by: Patrick East <east.patrick@gmail.com>
This tag will follow the current master branch and lines up with what
is in the docs as the `edge` version alias.
Signed-off-by: Patrick East <east.patrick@gmail.com>
This patch moves the check for unsafe built-ins from the server to the
compiler, so it can be used by the Go API as well as the HTTP API.
This was previously discussed in #1570.
Signed-off-by: Jasper Van der Jeugt <jasper@fugue.co>
This change updates the test command to use the fail explanation mode
by default. Since the test framework expects test rules to be
defined/true failure events are usually what is required to debug test
failures (e.g., some assertion in the test rule will Fail and that
will _likely_ be based on some failure in the policy under test.)
Hopefully this makes the test output a bit less verbose and more readable.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Also, refactor how the REPL prints debug state a bit to reduce
boilerplate for each possible explanation mode.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
These changes just rename the notes package so that it can be used
more generally. In addition, it adds another filter to show Fail
events and their lineage. The Fail filter will be exposed like the
Notes filter and will hopefully be useful in test scenarios.
This change is backwards incompatible but it's unlikely that anyone
outside of OPA was depending on the notes package. Updating
dependencies to use lineage.Notes instead of notes.Filter should be
easy enough if needed.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
These changes update the travis build to push the linux/amd64 binary
to an S3 bucket that allows public reads. This provides a simple
way for people to obtain and execute OPA locally which is useful when
Docker is not available (e.g., as is the case with Netlify.)
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Stan Lagun