Previously paths and queries were modelled with the same
attribute. This was going to cause headaches down the road for
decision log consumers that need to be able to deal with both kinds of
policy invocations.
As part of these changes, the decision logging test in the server has
been refactored to make it a bit more maintainable.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously OPA configuration for bundle downloading, status reporting, etc. had to be supplied in a configuration file on startup. With these changes, OPA can be configured to download a bundle that generates the OPA configuration. This allows OPA to boot with minimal configuration and dynamically update that configuration on-the-fly making it much easier to manage large deployments of OPAs for different use cases within the same system.
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
It's often useful to be able to create a hierarchical structure in one
shot in storage. Previously this functionality was implemented in the
server, but it's better off implemented in the storage package.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously we had constants defined for AST type names. These were used
in error messages in various places. The original goal was to make error
messages consistent, however, this approach made it difficult to locate
the source of the error in code.
Previously OPA only tracked query performance a high level (e.g., parse,
compile, eval latencies.) In some cases, it's necessary to instrument
lower level evaluation operations to understand performance. These
changes update the eval implementation to support instrumentation:
* Eval has been instrumented to record time taken for various core
operations like term plugging, reading from the store, rule lookup,
cache hits, etc.
* Rego package has been updated to support a simple rego.Instrument
operation that enables query instrumentation.
* REPL and server have been updated to expose simple interfaces to turn
on instrumentation.
* Diagnostic policy config "all" will enable instrumentation.
Instrumentation can be expensive (because it requires timing frequently
executed operations) so it should be treated as a debugging tool and not
enabled all of the time.
These changes expose the partial evaluation optimization through the
HTTP API. Clients can request the optimization by including a query
parameter and the server will lazily run partial evaluation and keep a
cache of the results for subsequent queries.
These changes modify topdown evaluation to use a binding list that
namespaces variables. This allows topdown to propagate partially ground
ref operands into child query evaluation.
These changes also prepare topdown evaluation to support a partial
evaluation mode.
With these changes, evaluation is no longer performed in two steps
(i.e., first pass of evaluating individual terms, second pass of
evaluating built-in expressions.) Instead, evaluation assumes queries
have been rewritten to eagerly evaluate refs and comprehension. This
way, ref and comprehension bindings do not have to be maintained
separately: they are handled by the normal variable binding list.
This commit contains some breaking changes to the topdown APIs,
namely...
1. Truth explanation has been removed. This feature was not used and the
tracing changes broke it. We can revisit in future if necessary.
2. Data indexing has been removed. Data indexing can be re-added in
future if necessary however it should be handled outside of topdown to
avoid potential memory leaks.
3. Built-in functions produce at-most-one output now. Functions that
used to produce multiple outputs (e.g., io.jwt.decode) can produce a
composite value if they need to.
Fixes#131
- These changes update the server to always report performance metrics
and the remote client address. The metrics are cheap to compute and
there is no significant reason to require more complex configuration
at this point.
- These changes update the runtime to allow callers to supply the
diagnostics buffer implementation. This way callers can hook up their
own sinks to the server's diagnostics.
Include an optional decision_id field in Data API responses. The
decision_id can be used for correlating application transactions with
policy decisions. The decision_id field is intended to be separate from
any kind of application tracing that OPA may eventually support.
In b23cb4e the compiler was changed to allow queries to refer to the
input document without the input document being defined. Those changes
did not remove all of the code associated with input errors.
These changes remove the remaining (dead) code associated with input
errors and also update the server to allow Data API POST requests that
do not specify an input document.
These changes update the Policy API responses to:
- Return empty objects for PUT/DELETE with optional support for ?metrics
- Return AST and raw/source for GET
Support for the ?source param has been removed as the raw/source version
is now included by default.
When the `pretty` parameter was supplied with a server request that also
requested explanations, the JSON representation of the explanation was
simply indented. JSON explanations are not human readable though, so
this patch updates the server to return a human readable string
representing explanations when the `pretty` parameter is true.
In some cases, services have existing webhook support that have
predefined request/response formats. Typically these webhooks perform an
HTTP POST against a configurable host, port, and path.
To ease integration of OPA with these services, we introduce a new API
version that represents the minimum required to perform policy
evaluation via HTTP.
As part of this change, the /policies/{id}/raw API has been removed in
favour of a "source" query parameter. When GET
/policies/{id}?source=true is received, the server returns the
raw/source of the policy.
Also, cleaned up handling of boolean query params.
Fixes#292
Torin Sandall