❗ We now parse rego metadata annotations by default.
Rule annotations now support a `labels` field. During policy eval,
labels from all successfully evaluated rules are collected and included
in each decision log entry as a top-level `rule_labels` array. Each
element preserves the label map from one evaluated rule. Exact
duplicates are omitted.
```rego
# METADATA
# labels:
# severity: low
# team: platform
allow if input.role == "admin"
```
The resulting decision log entry will contain:
```json
{"rule_labels": [{"severity": "low", "team": "platform"}]}
```
---------
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Rules can now be annotated with a metadata `id` field. When any
metadata `id` annotations are present in the rego (scope: rule), the IDs
of successfully evaluated rules are included in decision log events.
Additionally, the Data API supports a `?id` query parameter to
include evaluated rule IDs directly in the response payload.
```rego
# METADATA
# id: allow-admin
allow if input.role == "admin"
```
Modules containing `id` annotations will have metadata parsing enabled
automatically.
Fixes#2089
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This PR brings support for pluggable logging implementations via the logger plugin interface, which is based on Go's standard log/slog.Handler interface. This allows any slog.Handler implementation to be used as a logger plugin. Loggers can be referenced via the server.logger_plugin configuration option; and can also be used for decision logs. OPA includes a built-in file logger plugin (file_logger) that writes structured JSON logs with rotation support using lumberjack. Users can also implement and register custom logger plugins when building OPA.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This introduces a new trigger mode for the decision log plugin:
decision_logs.reporting.trigger=immediate
The immediate trigger mode will upload events as soon as enough events are received to hit the configured upload limit. If not enough events are received within the configured min-max delay, the events received so far are flushed and uploaded.
Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
decisions are being dropped because after an upload the limit is being set to the upload size instead of the buffer.
Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
Have done this some time in the past, but there was a few
new issues this would highlight now that we're on Go 1.24.
Mostly:
- Use `b.Loop()` in benchmarks
- Use `strings.SplitSeq` where possible
- Remove `omitempty` tag for types that can't be empty
Signed-off-by: Anders Eknert <anders@eknert.com>
In the encoder, if the event size equals the limit it was added to the buffer.
Instead return it to avoid unnecessary downsize step that could lead to an infinite loop.
Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
Updates the Bundle Plugins oneShot callback function signature used by Downloader, OCIDownloader, and fileLoader to return an error. This allows any issues in the callback function such as Rego parsing issues to be returned.
Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
refactors the original buffer implementation (now labelled as the size buffer) to implement the same interface as the event buffer type instead of being baked into the Plugin type. This helps create a clearer distinction between the two buffer types.
Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
Go 1.23 is no longer supported as per Go release policy.
Changes:
- Use Go v1.24.6 as the project SDK requirement
- Apply lint fixes for Go 1.24
- Fix "non-constant format string in call" issues as seen in CI.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
This commit adds an experimental "intermediate results" field to
decision logs, and provides some basic plumbing in the server package
for attaching the intermediate results of an eval to the request
context.
Co-authored-by: Teemu Koponen <koponen@styra.com>
Signed-off-by: Philip Conrad <philip@chariot-chaser.net>
This commit adds a new field to Decision Log entries, allowing batches
of decisions to be correlated together later.
Signed-off-by: Philip Conrad <philip@chariot-chaser.net>
`upload_size_limit_bytes` now only represents the compressed upload limit. Dropping the ND cache only happens if the event is too large after compression.
Signed-off-by: sspaink <sspaink@styra.com>
Since we don't use the HTTP API in Regal, I hadn't looked at this from
a performance POV before, and it was a fun side quest :)
A pretty decent reduction of the baseline cost for the most common
request type, v1/data POST. Changes:
- Add NoOp implementation of `Metrics` for when metrics aren't needed
- Cheaper `metrics.New` instantiation avoiding unnecessary lock
- Avoid cost of decision logging if decision logging isn't enabled
- Only call request.URL.Query() if URL.RawQuery isn't empty, avoiding
allocating an empty map with each request
While the target was v1/data POST handling, some of the changes above
have a positive impact on all or most handlers. All changes have been
run through the existing tests, and a new benchmark to measure the impact
of the fixes have been added, showing:
```
13063 ns/op 15162 B/op 195 allocs/op - main
12796 ns/op 14856 B/op 189 allocs/op - avoid r.URL.Query() when no query provided
12541 ns/op 14483 B/op 187 allocs/op - decisionLogger.Log early exit if not enabled
12133 ns/op 14235 B/op 180 allocs/op - get revisions and init logger only if needed
11098 ns/op 14207 B/op 180 allocs/op - more efficient metrics.New() (without locking)
10683 ns/op 13171 B/op 169 allocs/op - use no-op metrics implementation when metrics aren't requested
```
Even if the impact is pretty good, it's worth noting that most of the improvements
above are only seen when decision logging is turned off. While this is the common case
for development, it's not in production. Getting the baseline cost down is important still
as there should be no cost paid for features unused.
Signed-off-by: Anders Eknert <anders@styra.com>
This PR adds interning of strings representing common integer values,
which greatly speeds up "to string" operations on numbers, and updates
some built-ins commonly used for this to make use of interned values
where possible.
This is "light" version of a previous PR that did this more aggressively,
but also came with more caveats. Importantly, interning of new strings is
now never done at "runtime", but only allowed at init time. The API for
interning is marked experimental and should not relied upon by anyone
who expects a stable API.
Signed-off-by: Anders Eknert <anders@styra.com>
Following up on #7566, and now applying the more exciting
modernizations. fmt.Appendf was new to me! But especially
the contains checks are so much better IMHO. I have reviewed
all changes myself and did a few manual changes where it
became obvious that things could be improved a little further.
(the modernize analyzer still has some issues running against
OPA, and I have manually worked around those for the time being)
Signed-off-by: Anders Eknert <anders@styra.com>
If a status API is slow to respond it can cause OPA to be blocked writing to an unbuffered channel. This fixes it by using a buffered channel that never blocks but drops the oldest status update if full.
Signed-off-by: Sebastian Spaink <3441183+sspaink@users.noreply.github.com>
This new event-based buffer provides a performance improvement over
the existing buffer by reducing locks and allowing concurrent writes and uploads.
The buffer size is managed by number of individual events opposed to total bytes.
Signed-off-by: sspaink <sspaink@styra.com>
Brace yourselves! For there are many touched files here. No changes
in semantics however.
Spent a long time trying out the various optional rules gocritic
provides, and settled for a few of them. There are more I really
like, but that would take many hours to address across the codebase.
Perhaps others find gocritic too pedantic? If so, we can merge the
fixes without enabling the rule.
Signed-off-by: Anders Eknert <anders@styra.com>
And update code to conform to the rule.
- Replace unnecessary fmt.Sprintf with string concatenation
- Replace fmt.Sprint with more efficient strconv.Itoa
- Replace static fmt.Errorf calls with more efficient errors.New
Thanks @srenatus for pushing me down this rabbit hole!
Signed-off-by: Anders Eknert <anders@styra.com>
I pushed most of these fixed previously but these ones required
more work. Probably more work that it was worth, lol, but now the
work is done... and at least we can add this to the list of enabled
checks.
Fixes#7238
Signed-off-by: Anders Eknert <anders@styra.com>
All packages, except for `cmd` and `internal`, have been moved into a new `v1` root package.
Old packages are kept for backwards-compatibility reasons. All contained code is replaced with simple type aliases and proxy functions to `v1` implementations.
Old packages default to the Rego v0 syntax, new `v1` packages default to the Rego v1 syntax.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Stephan Renatus