Follow-up to #8900. Moves the gzip encoding and decoding config
validation off the Go `validateAndInjectDefaults` methods and onto
embedded Rego policies, injecting defaults and reporting value errors.
Each config registers its recognized options via
`config.RegisterConfigSpec` so unknown-option warnings live with the
owning struct. Field type validation stays in the Go decode step.
---------
Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
Part of #2745
Like most of his ideas, @anderseknert's suggestion to use Rego to
replace the `validateAndInjectDefaults` functions throughout the
codebase is another winner.
This PR starts the migration by replacing the top-level
`validateAndInjectDefaults` in `v1/config/config.go` with an embedded
policy, `validate.rego`. The policy injects the top-level defaults
(`default_decision`, `default_authorization_decision`, `labels`) and
reports unrecognized configuration options, so a typo such as
`decision_log` instead of `decision_logs` is logged as a warning at
startup rather than silently ignored.
It's evaluated in `ParseConfig` using the low-level `ast`/`topdown`
packages rather than the top-level `rego` package. This keeps `config`
off the heavy `rego → bundle → …` dependency web (which would otherwise
create import cycles as more packages' tests reach `config`), and we
don't need any of the `rego` package's conveniences here — it's one
module compiled once and a single query. The Rego unit tests run in CI
via `build/run-rego-tests.sh` (and locally with `make rego-test`).
This sets the foundation for the other plugin
`validateAndInjectDefaults` functions to migrate to Rego as well; where
the logic isn't too complicated it should be a fairly easy replacement.
At the moment all known keys live in `validate.rego` under `_specs` to
support the "warn on unrecognized options" check, but the
plugin-specific entries can move closer to each plugin as it migrates.
It would also be nice for `_specs` to be auto-generated somehow in the
future.
Supporting extension of config validation with custom policies is
something I'd like to follow up with, so keeping #2745 open for now.
I also think these policies could be reusable with
[java-opa-sdk](https://github.com/open-policy-agent/java-opa-sdk) 👀
Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
### Why the changes in this PR are needed?
`text/template`'s field evaluator (`text/template.(*state).evalField`,
`exec.go`) calls`reflect.Value.MethodByName` with a non-constant name.
The Go linker treats a reachable non-constant`MethodByName` as a signal
to disable **method-level dead-code elimination for the whole binary**
(see `cmd/link/internal/ld/deadcode.go` and golang/go#72895). Two OPA
code paths pull stdlib `text/template` into the reachable graph of
ordinary embedders:
1. **Compiler frontend** — `ast.Compiler.Compile → … →
gojsonschema.formatErrorDescription → text/template`. Reached
unconditionally by anything that compiles Rego.
2. **`strings.render_template` builtin** (`v1/topdown/template.go`) —
registered in the topdown builtin table, reachable in anything that
links Rego evaluation.
So an embedder of OPA's compiler/eval retains its entire reachable
method surface — a large binary-size regression, hundreds of MB in the
reporter's case (#7903). Both edges must go before the linker re-enables
method-level DCE for that embedder.
### What are the changes in this PR?
Vendor a self-contained, method-less copy of `text/template` under
`internal/methodlesstemplate` and point both call sites at it. **No
external dependency** (`go.mod`/`go.sum` unchanged).
- Copied verbatim from **Go 1.25.8**: `doc.go`, `exec.go`, `funcs.go`,
`option.go`, `template.go`, plus `internal/fmtsort/sort.go`. Go's BSD
`LICENSE` is preserved in the vendored directory and every file keeps
its `The Go Authors` copyright header.
- Stdlib `text/template/parse` is reused unchanged (the parser has no
`MethodByName`/`evalField` edge, so it does not defeat DCE).
- `helper.go` (`ParseFiles`/`ParseGlob`/`ParseFS`) is dropped — the OPA
call sites only need `New`/`Parse`/`Execute`, and nothing in the kept
files references it.
- **The only edit to the copied code** is removing the `MethodByName`
branch in `exec.go`'s `evalField` (method resolution on the data value).
Everything else is byte-identical, so re-syncing to a newer Go release
is a diff-and-reapply of that single branch removal.
- `internal/gojsonschema` (commit 1) and `v1/topdown` (commit 2) import
the vendored package. The gojsonschema engine is retained in full, so
`ErrorTemplateFuncs` (its `FuncMap` extension point) keeps working —
**no public symbol is removed**.
Rego values and gojsonschema `ErrorDetails` decode to
`map[string]any`/`[]any`/scalars, which have no methods, so removing
method resolution is a provable no-op for these callers.
### Notes to assist PR review:
- **Diff review tip**:
`doc.go`/`funcs.go`/`option.go`/`template.go`/`internal/fmtsort/sort.go`
are **byte-identical** to the Go 1.25.8 originals. Only `exec.go`
differs, in exactly two hunks: the `internal/fmtsort` → vendored import
path, and the removed `MethodByName` block (replaced by a comment
explaining the DCE rationale).
- **Fidelity — render_template**: the `rendertemplate` conformance cases
(incl. `complex` range/if/vars, `simpleint` `%v`, `missingkey` →
`<undefined>`) pass **unchanged**.
- **Fidelity — gojsonschema**: same engine (method-less),
validation-error output unchanged; existing `internal/gojsonschema` and
`v1/ast` tests pass.
- **Tests**: `TestNoStdlibTextTemplateImport` in both
`internal/gojsonschema` and `v1/topdown` scans every non-test file and
asserts none import stdlib `text/template`/`html/template`. `go build
./...`, `go vet ./...` OK; `go mod tidy` is a no-op.
- **Lint**: the vendored directory is added to the golangci-lint path
exclusions, mirroring the existing `internal/gojsonschema` precedent —
the copy is verbatim stdlib, and linting it against OPA's house rules
would force divergence from upstream Go (it trips ~31 stdlib-idiom
issues) and break the diff-and-reapply re-sync.
- **Attribution**: the vendored code is Go stdlib only (BSD, `The Go
Authors`); it contains no third-party/DataDog code.
### Further comments:
- **Scope**: this restores method-level DCE for embedders of OPA's
**compiler/eval**. The standalone `opa` binary additionally links
`v1/server`, which imports `html/template` (a wrapper over
`text/template`) — a separate, independent edge left as a follow-up.
Embedders that don't link the server (the common case) get the full win
from this PR.
- Root cause: golang/go#72895. Closes#7903 for compiler/eval embedders.
---------
Signed-off-by: Dick Childress <dick.childress@icearp.net>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This change adds a new flag for emitting plan bundles in the new protobuf wire format. `opa build --format=json|proto`. With `--format=proto`, the bundle contains `/plan.pb` and `/.manifest.pb` in place of `/plan.json`and `/.manifest`. Bundle Reader auto-detects both forms; mixed-format bundles are rejected at read, merge, and write time.
Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
The change replaces bytecodealliance/wasmtime-go/v44 (CGo) with
tetratelabs/wazero (pure Go)
- CGo eliminated — wazero is pure Go, so the whole internal/wasm/sdk
runtime no longer needs a C toolchain/cross-compilation story.
- The "env glue module" trick (glue.go) is the right solution to
wazero's constraint that a HostModuleBuilder can't export memory.
- Process-wide CompilationCache (sync.OnceValue): each unique policy is
compiled once per process, and discarded/re-instantiated VMs are cheap.
- Simplification in vm.go — dropping the ~25 closure fields (evalOneOff,
eval, heapPtrGet, …) in favor of mod.ExportedFunction(name) + a generic
call/callVoid/callOrCancel
- All tests pass (incl. internal/wasm/sdk/internal/wasm,
internal/wasm/sdk/opa). evalCompat for ABI 1.1 is retained.
----------
```
│ bf2bb5261c │ 13d2710058 │
│ sec/op │ sec/op vs base │
WASMColdStartTargets/topdown-16 112.8µ ± 1% 113.3µ ± 1% ~ (p=0.512 n=15)
WASMColdStartTargets/wasm-16 10.850m ± 1% 2.906m ± 1% -73.22% (p=0.000 n=15)
geomean 1.107m 573.9µ -48.14%
benchmark \ host local:tags=opa_wasm
vs base
WASMColdStartTargets/topdown ~
WASMColdStartTargets/wasm -73.22%
```
```
│ bf2bb5261c │ 13d2710058 │
│ sec/op │ sec/op vs base │
WasmRego-16 4.976µ ± 1% 3.546µ ± 3% -28.74% (p=0.000 n=15)
│ bf2bb5261c │ 13d2710058 │
│ B/op │ B/op vs base │
WasmRego-16 2.276Ki ± 0% 13.260Ki ± 0% +482.50% (p=0.000 n=15)
│ bf2bb5261c │ 13d2710058 │
│ allocs/op │ allocs/op vs base │
WasmRego-16 46.00 ± 0% 33.00 ± 0% -28.26% (p=0.000 n=15)
benchmark \ host local:tags=opa_wasm
vs base
WasmRego -28.74%
```
> [!NOTE]
> When running benchmarks here, be aware that the memory previously used
was invisible to the benchmark machinery -- it was on the other side of
the CGo divide 🙈Fixes#7557.
---------
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Contains simplified PE: expressions are plugged and saved, but not
optimized. PE optimization to follow in #8680
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Let's see if this works, it's a bit experimental at this point.
The twist comparred to how it's been done in EOPA (for example) is that
we're running the benchmarks post-merge, and report back if at the end
we find a failing check. This way, the PR goes green without having to
wait for the benchmarks, but there's still a connection between PR and
benchmark.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
* e2e/cli: add test for debug `print()` logging
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
* ci: add *.txtar to "golang_change_suffixes"
This is so that e2e/cli testscript definitions cause CI runs.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
---------
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
We have fixed most of the broken links! But,
https://github.com/open-policy-agent/opa/issues/8464 is mostly not useful now
as we are checking the docs site internal links which are already checked at
build time which is done in PRs.
This change makes two jobs, one for the repo, and one for the docs site. The
OPA domain is ignored for website checks.
Signed-off-by: Charlie Egan <charlie_egan@apple.com>
Disabled by default. To enable, `not` future keyword must be present in capabilities and imported into Rego module.
Implements: #8391
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
The testcase generator had a bug where very large numbers would be
parsed incorrectly, truncating the lower bits off their values.
This was discovered to be caused by the YAML library defaulting
to parsing all numeric values into floating point numbers, which
lose precision at larger sizes.
The fix was to provide the YAML unmarshaling function with the
appropriate equivalent of `(*json.Decoder).UseNumber()` at the
callsite. This causes the YAML library to use `json.Number`
types by default, just as we expect almost everywhere else in
Rego.
Signed-off-by: Philip Conrad <philip@chariot-chaser.net>
This PR contains fixes for all findings by the static analysis
tool zizmor, and reduces the attack surface available in our
GH Actions workflows by a decent margin.
The most notable change: our post-tag workflow now does not
use the actions cache, to prevent cache poisoning attacks.
This will drive up release publishing times, but eliminates
an attack vector on those releases.
Other changes:
- We also update all of our Slack alerting steps to use the
official slackapi/slack-github-action project, instead of the
archived project we were using before.
- A new `yaml` change detection category to has been added
to the `check-changes` job, allowing later jobs and steps
in the pull-request workflow to run conditionally on
YAML-based changes.
- An explicit linting job that runs the zizmor Github Actions
static analysis tool on the repo when YAML changes are
detected.
Signed-off-by: Philip Conrad <philip@chariot-chaser.net>
Due the way that wasmtime-go does its versioning, it seems to fly under
the radar of dependabot: that will never propose major version bumps, I
suppose.
Signed-off-by: Stephan Renatus <stephan@styra.com>
* build: bump go (1.24.6 -> 1.24.7)
https://groups.google.com/g/golang-announce/c/PtW9VW21NPs
* ci: run go stuff if .go-version changed
* Makefile: change debian base image
There is no 1.24.7-bullseye, because that distribution is EOL
* build: remove GOOS from Makefile call
Some change either in the debian trixie golang image, or in golang's
toolchain made this necessary: Prior to this commit, `GOOS` was set to
windows, causing a cross-build. We don't want a crossbuild for this
utility, we want to invoke the Linux binary (on the Linux host) to do
its job on a Windows binary (cross-built from the Linux host).
---------
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
The assertions are stricter now, e.g. we're also checking that nothing
is emitted to stderr.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This commit fixes an issue when upgrading codebases to OPA v1.7.0.
In PR #7797, we introduced the ability to provide "branding"
information in OPA commands and help messages, which would
allow easier customized OPA distributions in the future.
However, this changeset removed the public symbol `cmd.RootCommand`,
and required refactoring to use `cmd.Command`, which breaks automated
upgrades, such as those done by Dependabot.
This PR adds back the missing symbol, with the original/default "OPA"
branding provided. This should allow existing codebases to upgrade
without requiring any code changes.
Signed-off-by: Philip Conrad <philip@chariot-chaser.net>
This change allows users that build their own executable or "spin" of
OPA to give it a name, and have it reference itself properly in help
texts.
It's a vanity thing, but I think some people would appreciate it, hat
tip to the international association of pedants.
Signed-off-by: Stephan Renatus <stephan@styra.com>
Co-authored-by: kevinstyra <83973046+kevinstyra@users.noreply.github.com>
Closing the circle here, or something.
Not a lot of Rego used in OPA yet, but some in examples and tests. The little
there is should be linted though, and it'd be good if any new addition of policies got
linted by default. But more than anything, the "ignore configuration" provided here
avoids having developers seeing thousands of issues reported by Regal when they
open the OPA project in VS Code or their editor of choice.
Someone might want to look into un-ignoring the doc directory at some point, as it's
probably a good idea to have the docs follow best practices.
Signed-off-by: Anders Eknert <anders@styra.com>
The new command is based on generating JSON for docusaurus consumption
rather than markdown. This is less error prone as manipulation of
markdown is better contained.
Signed-off-by: Charlie Egan <charlie@styra.com>
Some things added on this branch:
- icons and client logos at known paths
- some spam redirects
- some redirects for docs/latest/foo /docs/foo
- redirects for the two ‘moved’ pages (k8s and envoy intros)
I am going to be monitoring the traffic this week on Netlify to make
sure we have a good coverage with redirects where needed.
We also now prompt users to file issues directly from the 404 pages
which should help flag things we miss too.
Brace yourselves! For there are many touched files here. No changes
in semantics however.
Spent a long time trying out the various optional rules gocritic
provides, and settled for a few of them. There are more I really
like, but that would take many hours to address across the codebase.
Perhaps others find gocritic too pedantic? If so, we can merge the
fixes without enabling the rule.
Signed-off-by: Anders Eknert <anders@styra.com>
And update code to conform to the rule.
- Replace unnecessary fmt.Sprintf with string concatenation
- Replace fmt.Sprint with more efficient strconv.Itoa
- Replace static fmt.Errorf calls with more efficient errors.New
Thanks @srenatus for pushing me down this rabbit hole!
Signed-off-by: Anders Eknert <anders@styra.com>
All packages, except for `cmd` and `internal`, have been moved into a new `v1` root package.
Old packages are kept for backwards-compatibility reasons. All contained code is replaced with simple type aliases and proxy functions to `v1` implementations.
Old packages default to the Rego v0 syntax, new `v1` packages default to the Rego v1 syntax.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Sebastian Spaink