And enable more staticcheck linters. I saw staticcheck failures
mentioned in another PR, so thought I'd check it out.
- `WriteString(fmt.Sprintf)` -> `fmt.Fprintf`
- Rewrite calls to deprecated `*Rule.Path()`
- Don't use `==` to compare `time.Time`
- Use inline ignores over config exclusions of paths
- Remove 'varcheck' ignores as no longer used
- Remove v0 topdown/graphql.go (!)
Signed-off-by: Anders Eknert <anders.eknert@apple.com>
- Bump golangci-lint -> 2.6.2
- Fix all `deprecatedComment` "notices should be in a dedicated paragraph, separated from the rest" reports
- Enable `appendCombine` and fix all "appendCombine: can combine chain of X appends into one" notices
- Enable `preferFprint` and fix the few reported issues
- Fix various issues reported only once or twice, like `zeroByteRepeat`
Signed-off-by: Anders Eknert <anders.eknert@apple.com>
All packages, except for `cmd` and `internal`, have been moved into a new `v1` root package.
Old packages are kept for backwards-compatibility reasons. All contained code is replaced with simple type aliases and proxy functions to `v1` implementations.
Old packages default to the Rego v0 syntax, new `v1` packages default to the Rego v1 syntax.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
This commit introduces a few major changes:
- (Breaking change) Limits now exist for maximum request body sizes.
- Buffers are preallocated for reading request bodies.
- Buffers are preallocated for decompressing request bodies.
- Gzip decoder instances are reused in a `sync.Pool` across requests.
The effect on garbage collection is dramatically fewer GC pauses, giving
a roughly 9% RPS improvement in load tests with gzipped request bodies.
For larger request sizes, the number of GC pauses is dramatically
reduced, although the peak pause time may increase by a few percent.
Implementation notes:
- The DecodingLimits handler enforces the max request body size both
through a Content-Length check, and a MaxBytesReader wrapper around
the payload.
- The DecodingLimits handler passes the gzip payload size limit down
using a context key.
Signed-off-by: Philip Conrad <philipaconrad@gmail.com>
The partial query parameter has been marked as deprecated since
v0.23.0. It's also removed from the docs since that time.
Fixes: #2266
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
Since "document missing or undefined" caused confusion, we can be
more helpful and report whether the default decision document is
missing, or whether it's there but undefined. I would probably have
preferred for undefined to just result in an empty/undefined result,
but that'd be quite a breaking change at this point in time, so
hopefully this can at least make things a little less convoluted.
Fixes#5344
Signed-off-by: Anders Eknert <anders@eknert.com>
This would be useful for us for two immediate use cases:
1. Show how and why rules failed in more detail in verbose tooling,
we can show the unification happening step by step.
2. We can trace which parts of the input document were used, if we
add `Location` info those terms.
However, I think it's generally useful for debugging tools.
This increases verbosity in the explain logs, so we decided to add a new explain
mode `debug` in addition to the existing `full`, `notes`, `fails`, `off` modes.
This can be set using the `--explain=debug` flag on the CLI, or by using `trace
debug` in the REPL.
Signed-off-by: Jasper Van der Jeugt <m@jaspervdj.be>
This change adds an api_usage_warning when input attribute is missing in POST requests to /v1/data
Fixes: #4386
Signed-off-by: Alam <afaaqalam@gmail.com>
This change enables consumers of Compile API to specify packages
those should not be inlined in partial evaluation response.
Fixes: #4357
Signed-off-by: skosunda <skosunda@adobe.com>
This commit adds a new server endpoint for pulling the status
information from the running OPA. Normally status is pushed by OPA to
remote locations but in some cases users may need to pull it.
The docs changes move the config and health API sections up into the
right location in the REST API page.
Fixes#4089
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
* Wrap entire plugin state get/set operation with mutex
* Tweak error messages returned to clients
* Tweak docs explaining use case for custom checks
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
- Added a new querystring param to the /health route: `exclude-plugin`
- Can be specified multiple times
- Value is the name of a plugin that should be excluded from the OK
status check performed when the `plugins` qs param is specified
Fixes#3713
Signed-off-by: Grant Shively <gshively@godaddy.com>
This commit just tweaks the new v1/config response to include a
wrapper that nests the config object under a "result" key in the
response. This makes GET v1/config consistent with other OPA APIs and
future-proofs the API to some extent (e.g., if we need to include
additional fields in the future, they do not necessarily have to be
included in the config object itself.)
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This commit removes the deprecated watch package and server
feature. This feature was never adopted and since the implementation
was not incremental, it would inevitably encounter
performance/scalability issues.
Fixes#2265
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Recently we improved the trace pretty printing to include location
information on events. Unless there's a good reason we should use this
tracer printing throughout.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This deprecates `/health?bundle` in favor of (the plural)
`/health?bundles`.
Both have the same API result, taking into account *all* bundle
statuses. This just lessen any confusion with regards to whether
`bundle` means a single bundle or all. It fits better with the new
`plugins` option.
Signed-off-by: Patrick East <east.patrick@gmail.com>
This commit removes the deprecated diagnostic feature from the
server. The feature has been deprecated since November 2018 and it was
essentially unused at the time so it should be safe to
remove. Removing the diagnostic support from the server saves having
to perform an extra policy evaluation in the server.
Once the buffer is removed from the runtime.Params struct the related
issue can be closed (there is still one known user of that so it has
been left intact for backwards compatibility.)
Ref #1052
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This change brings in support for multiple bundles to be downloaded
and activated OPA.
This is enabled by using the new config option `bundles` to define
the bundles, and deprecates the older `bundle` option.
The new `bundles` keyword and structure is propagated through to the
decision logs, status API, provenance, stored manifests, etc. Check
out the doc changes for all the updated structures.
That being said any existing configuration using `bundle` will *not*
see the new structure, everything is intended to be backwards
compatible (almost to a fault).
Fixes: #721
Signed-off-by: Patrick East <east.patrick@gmail.com>
This just tweaks the error string format introduced in #1530 to be
consistent with other error strings in OPA (e.g., topdown, ast, etc.)
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
There is a new parameter for /health REST API which will include the
Configured bundle activation in the response. Example:
GET /health?bundle=true HTTP/1.1
Without the parameter the behavior stays the same, with it the server
will respond with 500’s until the status has been updated with an
activation time.
The docs for kubernetes ready probe has been updated to show this as
it makes for a better ready check than the original behavior when
remote bundles are being used.
Fixes: #1153
Signed-off-by: Patrick East <east.patrick@gmail.com>
First of several commits that define a new query
parameter named 'provenance'.
This query parameter will return version/build information
in addition to the result to the caller for data REST calls.
Signed-off-by: Peter W. Morreale <pmorreale@statestreet.com>
Previously paths and queries were modelled with the same
attribute. This was going to cause headaches down the road for
decision log consumers that need to be able to deal with both kinds of
policy invocations.
As part of these changes, the decision logging test in the server has
been refactored to make it a bit more maintainable.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously OPA configuration for bundle downloading, status reporting, etc. had to be supplied in a configuration file on startup. With these changes, OPA can be configured to download a bundle that generates the OPA configuration. This allows OPA to boot with minimal configuration and dynamically update that configuration on-the-fly making it much easier to manage large deployments of OPAs for different use cases within the same system.
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
It's often useful to be able to create a hierarchical structure in one
shot in storage. Previously this functionality was implemented in the
server, but it's better off implemented in the storage package.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Previously we had constants defined for AST type names. These were used
in error messages in various places. The original goal was to make error
messages consistent, however, this approach made it difficult to locate
the source of the error in code.
Previously OPA only tracked query performance a high level (e.g., parse,
compile, eval latencies.) In some cases, it's necessary to instrument
lower level evaluation operations to understand performance. These
changes update the eval implementation to support instrumentation:
* Eval has been instrumented to record time taken for various core
operations like term plugging, reading from the store, rule lookup,
cache hits, etc.
* Rego package has been updated to support a simple rego.Instrument
operation that enables query instrumentation.
* REPL and server have been updated to expose simple interfaces to turn
on instrumentation.
* Diagnostic policy config "all" will enable instrumentation.
Instrumentation can be expensive (because it requires timing frequently
executed operations) so it should be treated as a debugging tool and not
enabled all of the time.
These changes expose the partial evaluation optimization through the
HTTP API. Clients can request the optimization by including a query
parameter and the server will lazily run partial evaluation and keep a
cache of the results for subsequent queries.
These changes modify topdown evaluation to use a binding list that
namespaces variables. This allows topdown to propagate partially ground
ref operands into child query evaluation.
These changes also prepare topdown evaluation to support a partial
evaluation mode.
With these changes, evaluation is no longer performed in two steps
(i.e., first pass of evaluating individual terms, second pass of
evaluating built-in expressions.) Instead, evaluation assumes queries
have been rewritten to eagerly evaluate refs and comprehension. This
way, ref and comprehension bindings do not have to be maintained
separately: they are handled by the normal variable binding list.
This commit contains some breaking changes to the topdown APIs,
namely...
1. Truth explanation has been removed. This feature was not used and the
tracing changes broke it. We can revisit in future if necessary.
2. Data indexing has been removed. Data indexing can be re-added in
future if necessary however it should be handled outside of topdown to
avoid potential memory leaks.
3. Built-in functions produce at-most-one output now. Functions that
used to produce multiple outputs (e.g., io.jwt.decode) can produce a
composite value if they need to.
Fixes#131
- These changes update the server to always report performance metrics
and the remote client address. The metrics are cheap to compute and
there is no significant reason to require more complex configuration
at this point.
- These changes update the runtime to allow callers to supply the
diagnostics buffer implementation. This way callers can hook up their
own sinks to the server's diagnostics.
Include an optional decision_id field in Data API responses. The
decision_id can be used for correlating application transactions with
policy decisions. The decision_id field is intended to be separate from
any kind of application tracing that OPA may eventually support.
In b23cb4e the compiler was changed to allow queries to refer to the
input document without the input document being defined. Those changes
did not remove all of the code associated with input errors.
These changes remove the remaining (dead) code associated with input
errors and also update the server to allow Data API POST requests that
do not specify an input document.
These changes update the Policy API responses to:
- Return empty objects for PUT/DELETE with optional support for ?metrics
- Return AST and raw/source for GET
Support for the ?source param has been removed as the raw/source version
is now included by default.
When the `pretty` parameter was supplied with a server request that also
requested explanations, the JSON representation of the explanation was
simply indented. JSON explanations are not human readable though, so
this patch updates the server to return a human readable string
representing explanations when the `pretty` parameter is true.
Anders Eknert