vendor a method-less text/template to restore whole-binary linker DCE (#8844)

### Why the changes in this PR are needed?

`text/template`'s field evaluator (`text/template.(*state).evalField`,
`exec.go`) calls`reflect.Value.MethodByName` with a non-constant name.
The Go linker treats a reachable non-constant`MethodByName` as a signal
to disable **method-level dead-code elimination for the whole binary**
(see `cmd/link/internal/ld/deadcode.go` and golang/go#72895). Two OPA
code paths pull stdlib `text/template` into the reachable graph of
ordinary embedders:

1. **Compiler frontend** — `ast.Compiler.Compile → … →
gojsonschema.formatErrorDescription → text/template`. Reached
unconditionally by anything that compiles Rego.
2. **`strings.render_template` builtin** (`v1/topdown/template.go`) —
registered in the topdown builtin table, reachable in anything that
links Rego evaluation.

So an embedder of OPA's compiler/eval retains its entire reachable
method surface — a large binary-size regression, hundreds of MB in the
reporter's case (#7903). Both edges must go before the linker re-enables
method-level DCE for that embedder.

### What are the changes in this PR?

Vendor a self-contained, method-less copy of `text/template` under
`internal/methodlesstemplate` and point both call sites at it. **No
external dependency** (`go.mod`/`go.sum` unchanged).

- Copied verbatim from **Go 1.25.8**: `doc.go`, `exec.go`, `funcs.go`,
`option.go`, `template.go`, plus `internal/fmtsort/sort.go`. Go's BSD
`LICENSE` is preserved in the vendored directory and every file keeps
its `The Go Authors` copyright header.
- Stdlib `text/template/parse` is reused unchanged (the parser has no
`MethodByName`/`evalField` edge, so it does not defeat DCE).
- `helper.go` (`ParseFiles`/`ParseGlob`/`ParseFS`) is dropped — the OPA
call sites only need `New`/`Parse`/`Execute`, and nothing in the kept
files references it.
- **The only edit to the copied code** is removing the `MethodByName`
branch in `exec.go`'s `evalField` (method resolution on the data value).
Everything else is byte-identical, so re-syncing to a newer Go release
is a diff-and-reapply of that single branch removal.
- `internal/gojsonschema` (commit 1) and `v1/topdown` (commit 2) import
the vendored package. The gojsonschema engine is retained in full, so
`ErrorTemplateFuncs` (its `FuncMap` extension point) keeps working —
**no public symbol is removed**.

Rego values and gojsonschema `ErrorDetails` decode to
`map[string]any`/`[]any`/scalars, which have no methods, so removing
method resolution is a provable no-op for these callers.

### Notes to assist PR review:

- **Diff review tip**:
`doc.go`/`funcs.go`/`option.go`/`template.go`/`internal/fmtsort/sort.go`
are **byte-identical** to the Go 1.25.8 originals. Only `exec.go`
differs, in exactly two hunks: the `internal/fmtsort` → vendored import
path, and the removed `MethodByName` block (replaced by a comment
explaining the DCE rationale).
- **Fidelity — render_template**: the `rendertemplate` conformance cases
(incl. `complex` range/if/vars, `simpleint` `%v`, `missingkey` →
`<undefined>`) pass **unchanged**.
- **Fidelity — gojsonschema**: same engine (method-less),
validation-error output unchanged; existing `internal/gojsonschema` and
`v1/ast` tests pass.
- **Tests**: `TestNoStdlibTextTemplateImport` in both
`internal/gojsonschema` and `v1/topdown` scans every non-test file and
asserts none import stdlib `text/template`/`html/template`. `go build
./...`, `go vet ./...` OK; `go mod tidy` is a no-op.
- **Lint**: the vendored directory is added to the golangci-lint path
exclusions, mirroring the existing `internal/gojsonschema` precedent —
the copy is verbatim stdlib, and linting it against OPA's house rules
would force divergence from upstream Go (it trips ~31 stdlib-idiom
issues) and break the diff-and-reapply re-sync.
- **Attribution**: the vendored code is Go stdlib only (BSD, `The Go
Authors`); it contains no third-party/DataDog code.

### Further comments:

- **Scope**: this restores method-level DCE for embedders of OPA's
**compiler/eval**. The standalone `opa` binary additionally links
`v1/server`, which imports `html/template` (a wrapper over
`text/template`) — a separate, independent edge left as a follow-up.
Embedders that don't link the server (the common case) get the full win
from this PR.
- Root cause: golang/go#72895. Closes #7903 for compiler/eval embedders.

---------

Signed-off-by: Dick Childress <dick.childress@icearp.net>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
rchildress87
2026-07-07 15:08:19 -04:00
committed by GitHub
parent 89afc26b87
commit 81589c1244
15 changed files with 3167 additions and 3 deletions
+1
View File
@@ -102,6 +102,7 @@ linters:
path: _test\.go
paths:
- internal/gojsonschema
- internal/methodlesstemplate
- node_modules
issues:
# don't hide issues in CI runs because they are the same type
+121
View File
@@ -0,0 +1,121 @@
#!/usr/bin/env bash
# Regenerate internal/methodlesstemplate from the Go stdlib text/template.
#
# internal/methodlesstemplate is a verbatim copy of the standard library's
# text/template with exactly ONE behavioral edit: the method-calls-on-data
# branch in exec.go's evalField (reflect.Value.MethodByName) is removed. A
# reachable non-constant MethodByName forces the Go linker to disable
# method-level dead-code elimination for the whole binary (golang/go#72895), so
# eliding it is what lets OPA embedders shed the unused reflected method surface
# (#7903). Rego values and gojsonschema ErrorDetails decode to
# map[string]any/[]any/scalars, which have no methods, so the elision is a no-op.
#
# This script re-syncs that copy to whatever Go toolchain `go` resolves. Bump the
# vendored version by running it under a newer toolchain, e.g.:
# GOTOOLCHAIN=go1.26.0 build/regen-methodless-template.sh
#
# It copies the stdlib files verbatim, then re-applies the two-hunk exec.go edit
# (below) via `git apply`. helper.go and *_test.go are intentionally NOT vendored
# (OPA only needs New/Parse/Execute), and text/template/parse is reused from the
# stdlib via its normal import, so it is not copied here.
#
# If `git apply` fails, the stdlib changed the import block or the evalField
# region this edit targets: re-derive the elision by hand and update the PATCH
# heredoc at the bottom of this script.
set -euo pipefail
REPO_ROOT="$(git rev-parse --show-toplevel)"
cd "$REPO_ROOT"
if [[ "$(go list -m 2>/dev/null)" != "github.com/open-policy-agent/opa" ]]; then
echo "error: must run inside the open-policy-agent/opa module" >&2
exit 1
fi
GOROOT="$(go env GOROOT)"
TMPL_SRC="$GOROOT/src/text/template"
PKG="internal/methodlesstemplate"
echo "Regenerating $PKG from $(go version)"
# Verbatim stdlib copies. exec.go is patched below; the rest are byte-identical.
mkdir -p "$PKG/internal/fmtsort"
for f in doc.go exec.go funcs.go option.go template.go; do
cp -f "$TMPL_SRC/$f" "$PKG/$f"
done
cp -f "$GOROOT/src/internal/fmtsort/sort.go" "$PKG/internal/fmtsort/sort.go"
cp -f "$GOROOT/LICENSE" "$PKG/LICENSE"
# Re-apply the single intended edit to exec.go: swap the internal/fmtsort import
# for the vendored one, and remove the MethodByName data-method branch.
# --recount tolerates line-number drift from unrelated stdlib changes; it fails
# only if the patched context text itself changed.
if ! git apply --recount <<'PATCH'
--- a/internal/methodlesstemplate/exec.go
+++ b/internal/methodlesstemplate/exec.go
@@ -7,12 +7,13 @@
import (
"errors"
"fmt"
- "internal/fmtsort"
"io"
"reflect"
"runtime"
"strings"
"text/template/parse"
+
+ "github.com/open-policy-agent/opa/internal/methodlesstemplate/internal/fmtsort"
)
// maxExecDepth specifies the maximum stack depth of templates within
@@ -689,21 +690,18 @@
typ := receiver.Type()
receiver, isNil := indirect(receiver)
if receiver.Kind() == reflect.Interface && isNil {
- // Calling a method on a nil interface can't work. The
- // MethodByName method call below would panic.
+ // Indexing into a nil interface can't work.
s.errorf("nil pointer evaluating %s.%s", typ, fieldName)
return zero
}
- // Unless it's an interface, need to get to a value of type *T to guarantee
- // we see all methods of T and *T.
- ptr := receiver
- if ptr.Kind() != reflect.Interface && ptr.Kind() != reflect.Pointer && ptr.CanAddr() {
- ptr = ptr.Addr()
- }
- if method := ptr.MethodByName(fieldName); method.IsValid() {
- return s.evalCall(dot, method, false, node, fieldName, args, final)
- }
+ // OPA-DCE (#7903): the upstream text/template resolves methods on the data
+ // value here via reflect.Value.MethodByName. A reachable non-constant
+ // MethodByName disables the Go linker's method-level dead-code elimination
+ // binary-wide (golang/go#72895), so that branch is deliberately removed.
+ // Rego values (and gojsonschema ErrorDetails) decode to
+ // map[string]any/[]any/scalars, which have no methods, so field/element
+ // resolution below is the only path OPA's callers need.
hasArgs := len(args) > 1 || !isMissing(final)
// It's not a method; must be a field of a struct or an element of a map.
switch receiver.Kind() {
PATCH
then
echo "error: could not apply the method-elision patch to exec.go." >&2
echo " The stdlib import block or evalField region changed; re-derive" >&2
echo " the edit by hand and update the PATCH heredoc in $0." >&2
exit 1
fi
gofmt -w "$PKG"
# Fidelity guard: the whole point is that no reflect method-call on data survives.
if grep -rn '\.MethodByName(' "$PKG"; then
echo "error: $PKG still contains a .MethodByName( call after patching." >&2
exit 1
fi
go build ./"$PKG"/...
echo "Done. Vendored $PKG from $(go version)."
echo "Review 'git diff $PKG' before committing; run 'make go-test' for full verification."
+8 -1
View File
@@ -4,7 +4,14 @@ package gojsonschema
import (
"bytes"
"sync"
"text/template"
// A method-less copy of text/template (see internal/methodlesstemplate). Locale
// format strings expand only simple {{.field}} placeholders over ErrorDetails
// (map[string]any), which has no methods, so eliding method calls is a no-op
// here; it keeps text/template's evalField MethodByName off the reachable
// graph, which otherwise disables the Go linker's method-level dead-code
// elimination binary-wide (golang/go#72895, #7903).
template "github.com/open-policy-agent/opa/internal/methodlesstemplate"
)
var errorTemplates = errorTemplate{template.New("errors-new"), sync.RWMutex{}}
+46
View File
@@ -0,0 +1,46 @@
package gojsonschema
import (
"go/parser"
"go/token"
"os"
"strings"
"testing"
)
// TestNoStdlibTextTemplateImport guards the linker dead-code-elimination fix
// (#7903): the error formatter uses a method-less template package, never stdlib
// text/template or html/template. Their evalField reaches
// reflect.Value.MethodByName, whose reachability disables method-level DCE for
// the whole binary of every OPA embedder (golang/go#72895). gojsonschema is
// reached unconditionally from ast.Compiler.Compile, so a stdlib import here
// keeps the trigger live for every compiler embedder. Scans every non-test file
// in the package so the guard holds even if the import moves.
func TestNoStdlibTextTemplateImport(t *testing.T) {
entries, err := os.ReadDir(".")
if err != nil {
t.Fatal(err)
}
fset := token.NewFileSet()
scanned := 0
for _, e := range entries {
name := e.Name()
if e.IsDir() || !strings.HasSuffix(name, ".go") || strings.HasSuffix(name, "_test.go") {
continue
}
f, err := parser.ParseFile(fset, name, nil, parser.ImportsOnly)
if err != nil {
t.Fatalf("parse %s: %v", name, err)
}
scanned++
for _, imp := range f.Imports {
switch strings.Trim(imp.Path.Value, `"`) {
case "text/template", "html/template":
t.Errorf("%s imports %s — reintroduces the linker DCE-defeat trigger", name, imp.Path.Value)
}
}
}
if scanned == 0 {
t.Fatal("scanned no package files; test ran from the wrong directory")
}
}
+4 -1
View File
@@ -31,7 +31,10 @@ import (
"errors"
"math/big"
"regexp"
"text/template"
// Method-less copy of text/template; see the import note in errors.go
// (golang/go#72895, #7903). ErrorTemplateFuncs below is its FuncMap.
template "github.com/open-policy-agent/opa/internal/methodlesstemplate"
"github.com/xeipuuv/gojsonreference"
)
+27
View File
@@ -0,0 +1,27 @@
Copyright 2009 The Go Authors.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of Google LLC nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+28
View File
@@ -0,0 +1,28 @@
# methodlesstemplate
A vendored copy of the Go standard library's `text/template`, with **one** behavioral change: the
method-calls-on-data branch in `exec.go`'s `evalField` (`reflect.Value.MethodByName`) is removed.
A reachable, non-constant `reflect.Value.MethodByName` makes the Go linker disable method-level
dead-code elimination for the **entire binary** (golang/go#72895). OPA reaches `text/template` from
its compiler (schema errors) and the `strings.render_template` builtin, so that one edge retains the
full reflected method surface of every embedder — a large binary-size regression (#7903). Rego
values and gojsonschema `ErrorDetails` decode to `map[string]any` / `[]any` / scalars, which have no
methods, so eliding the data-method lookup is a behavioral no-op while restoring DCE.
Only `exec.go` differs from the upstream stdlib. `doc.go`, `funcs.go`, `option.go`, `template.go`,
and `internal/fmtsort/sort.go` are byte-identical to their Go release; `text/template/parse` is
reused via its normal import. `helper.go` (ParseFiles/ParseGlob/ParseFS) is intentionally not
vendored. Go's BSD `LICENSE` and per-file copyright headers are preserved.
## Regenerating
Do not hand-edit these files. To re-sync to a new Go release, run under the target toolchain:
```
GOTOOLCHAIN=go1.26.0 build/regen-methodless-template.sh
```
The script copies the stdlib files verbatim and re-applies the single method-elision edit. If the
edit no longer applies (the stdlib changed that region), the script fails and the elision must be
re-derived and the patch in the script updated.
+502
View File
@@ -0,0 +1,502 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
/*
Package template implements data-driven templates for generating textual output.
To generate HTML output, see [html/template], which has the same interface
as this package but automatically secures HTML output against certain attacks.
Templates are executed by applying them to a data structure. Annotations in the
template refer to elements of the data structure (typically a field of a struct
or a key in a map) to control execution and derive values to be displayed.
Execution of the template walks the structure and sets the cursor, represented
by a period '.' and called "dot", to the value at the current location in the
structure as execution proceeds.
The security model used by this package assumes that template authors are
trusted. The package does not auto-escape output, so injecting code into
a template can lead to arbitrary code execution if the template is executed
by an untrusted source.
The input text for a template is UTF-8-encoded text in any format.
"Actions"--data evaluations or control structures--are delimited by
"{{" and "}}"; all text outside actions is copied to the output unchanged.
Once parsed, a template may be executed safely in parallel, although if parallel
executions share a Writer the output may be interleaved.
Here is a trivial example that prints "17 items are made of wool".
type Inventory struct {
Material string
Count uint
}
sweaters := Inventory{"wool", 17}
tmpl, err := template.New("test").Parse("{{.Count}} items are made of {{.Material}}")
if err != nil { panic(err) }
err = tmpl.Execute(os.Stdout, sweaters)
if err != nil { panic(err) }
More intricate examples appear below.
Text and spaces
By default, all text between actions is copied verbatim when the template is
executed. For example, the string " items are made of " in the example above
appears on standard output when the program is run.
However, to aid in formatting template source code, if an action's left
delimiter (by default "{{") is followed immediately by a minus sign and white
space, all trailing white space is trimmed from the immediately preceding text.
Similarly, if the right delimiter ("}}") is preceded by white space and a minus
sign, all leading white space is trimmed from the immediately following text.
In these trim markers, the white space must be present:
"{{- 3}}" is like "{{3}}" but trims the immediately preceding text, while
"{{-3}}" parses as an action containing the number -3.
For instance, when executing the template whose source is
"{{23 -}} < {{- 45}}"
the generated output would be
"23<45"
For this trimming, the definition of white space characters is the same as in Go:
space, horizontal tab, carriage return, and newline.
Actions
Here is the list of actions. "Arguments" and "pipelines" are evaluations of
data, defined in detail in the corresponding sections that follow.
*/
// {{/* a comment */}}
// {{- /* a comment with white space trimmed from preceding and following text */ -}}
// A comment; discarded. May contain newlines.
// Comments do not nest and must start and end at the
// delimiters, as shown here.
/*
{{pipeline}}
The default textual representation (the same as would be
printed by fmt.Print) of the value of the pipeline is copied
to the output.
{{if pipeline}} T1 {{end}}
If the value of the pipeline is empty, no output is generated;
otherwise, T1 is executed. The empty values are false, 0, any
nil pointer or interface value, and any array, slice, map, or
string of length zero.
Dot is unaffected.
{{if pipeline}} T1 {{else}} T0 {{end}}
If the value of the pipeline is empty, T0 is executed;
otherwise, T1 is executed. Dot is unaffected.
{{if pipeline}} T1 {{else if pipeline}} T0 {{end}}
To simplify the appearance of if-else chains, the else action
of an if may include another if directly; the effect is exactly
the same as writing
{{if pipeline}} T1 {{else}}{{if pipeline}} T0 {{end}}{{end}}
{{range pipeline}} T1 {{end}}
The value of the pipeline must be an array, slice, map, iter.Seq,
iter.Seq2, integer or channel.
If the value of the pipeline has length zero, nothing is output;
otherwise, dot is set to the successive elements of the array,
slice, or map and T1 is executed. If the value is a map and the
keys are of basic type with a defined order, the elements will be
visited in sorted key order.
{{range pipeline}} T1 {{else}} T0 {{end}}
The value of the pipeline must be an array, slice, map, iter.Seq,
iter.Seq2, integer or channel.
If the value of the pipeline has length zero, dot is unaffected and
T0 is executed; otherwise, dot is set to the successive elements
of the array, slice, or map and T1 is executed.
{{break}}
The innermost {{range pipeline}} loop is ended early, stopping the
current iteration and bypassing all remaining iterations.
{{continue}}
The current iteration of the innermost {{range pipeline}} loop is
stopped, and the loop starts the next iteration.
{{template "name"}}
The template with the specified name is executed with nil data.
{{template "name" pipeline}}
The template with the specified name is executed with dot set
to the value of the pipeline.
{{block "name" pipeline}} T1 {{end}}
A block is shorthand for defining a template
{{define "name"}} T1 {{end}}
and then executing it in place
{{template "name" pipeline}}
The typical use is to define a set of root templates that are
then customized by redefining the block templates within.
{{with pipeline}} T1 {{end}}
If the value of the pipeline is empty, no output is generated;
otherwise, dot is set to the value of the pipeline and T1 is
executed.
{{with pipeline}} T1 {{else}} T0 {{end}}
If the value of the pipeline is empty, dot is unaffected and T0
is executed; otherwise, dot is set to the value of the pipeline
and T1 is executed.
{{with pipeline}} T1 {{else with pipeline}} T0 {{end}}
To simplify the appearance of with-else chains, the else action
of a with may include another with directly; the effect is exactly
the same as writing
{{with pipeline}} T1 {{else}}{{with pipeline}} T0 {{end}}{{end}}
Arguments
An argument is a simple value, denoted by one of the following.
- A boolean, string, character, integer, floating-point, imaginary
or complex constant in Go syntax. These behave like Go's untyped
constants. Note that, as in Go, whether a large integer constant
overflows when assigned or passed to a function can depend on whether
the host machine's ints are 32 or 64 bits.
- The keyword nil, representing an untyped Go nil.
- The character '.' (period):
.
The result is the value of dot.
- A variable name, which is a (possibly empty) alphanumeric string
preceded by a dollar sign, such as
$piOver2
or
$
The result is the value of the variable.
Variables are described below.
- The name of a field of the data, which must be a struct, preceded
by a period, such as
.Field
The result is the value of the field. Field invocations may be
chained:
.Field1.Field2
Fields can also be evaluated on variables, including chaining:
$x.Field1.Field2
- The name of a key of the data, which must be a map, preceded
by a period, such as
.Key
The result is the map element value indexed by the key.
Key invocations may be chained and combined with fields to any
depth:
.Field1.Key1.Field2.Key2
Although the key must be an alphanumeric identifier, unlike with
field names they do not need to start with an upper case letter.
Keys can also be evaluated on variables, including chaining:
$x.key1.key2
- The name of a niladic method of the data, preceded by a period,
such as
.Method
The result is the value of invoking the method with dot as the
receiver, dot.Method(). Such a method must have one return value (of
any type) or two return values, the second of which is an error.
If it has two and the returned error is non-nil, execution terminates
and an error is returned to the caller as the value of Execute.
Method invocations may be chained and combined with fields and keys
to any depth:
.Field1.Key1.Method1.Field2.Key2.Method2
Methods can also be evaluated on variables, including chaining:
$x.Method1.Field
- The name of a niladic function, such as
fun
The result is the value of invoking the function, fun(). The return
types and values behave as in methods. Functions and function
names are described below.
- A parenthesized instance of one the above, for grouping. The result
may be accessed by a field or map key invocation.
print (.F1 arg1) (.F2 arg2)
(.StructValuedMethod "arg").Field
Arguments may evaluate to any type; if they are pointers the implementation
automatically indirects to the base type when required.
If an evaluation yields a function value, such as a function-valued
field of a struct, the function is not invoked automatically, but it
can be used as a truth value for an if action and the like. To invoke
it, use the call function, defined below.
Pipelines
A pipeline is a possibly chained sequence of "commands". A command is a simple
value (argument) or a function or method call, possibly with multiple arguments:
Argument
The result is the value of evaluating the argument.
.Method [Argument...]
The method can be alone or the last element of a chain but,
unlike methods in the middle of a chain, it can take arguments.
The result is the value of calling the method with the
arguments:
dot.Method(Argument1, etc.)
functionName [Argument...]
The result is the value of calling the function associated
with the name:
function(Argument1, etc.)
Functions and function names are described below.
A pipeline may be "chained" by separating a sequence of commands with pipeline
characters '|'. In a chained pipeline, the result of each command is
passed as the last argument of the following command. The output of the final
command in the pipeline is the value of the pipeline.
The output of a command will be either one value or two values, the second of
which has type error. If that second value is present and evaluates to
non-nil, execution terminates and the error is returned to the caller of
Execute.
Variables
A pipeline inside an action may initialize a variable to capture the result.
The initialization has syntax
$variable := pipeline
where $variable is the name of the variable. An action that declares a
variable produces no output.
Variables previously declared can also be assigned, using the syntax
$variable = pipeline
If a "range" action initializes a variable, the variable is set to the
successive elements of the iteration. Also, a "range" may declare two
variables, separated by a comma:
range $index, $element := pipeline
in which case $index and $element are set to the successive values of the
array/slice index or map key and element, respectively. Note that if there is
only one variable, it is assigned the element; this is opposite to the
convention in Go range clauses.
A variable's scope extends to the "end" action of the control structure ("if",
"with", or "range") in which it is declared, or to the end of the template if
there is no such control structure. A template invocation does not inherit
variables from the point of its invocation.
When execution begins, $ is set to the data argument passed to Execute, that is,
to the starting value of dot.
Examples
Here are some example one-line templates demonstrating pipelines and variables.
All produce the quoted word "output":
{{"\"output\""}}
A string constant.
{{`"output"`}}
A raw string constant.
{{printf "%q" "output"}}
A function call.
{{"output" | printf "%q"}}
A function call whose final argument comes from the previous
command.
{{printf "%q" (print "out" "put")}}
A parenthesized argument.
{{"put" | printf "%s%s" "out" | printf "%q"}}
A more elaborate call.
{{"output" | printf "%s" | printf "%q"}}
A longer chain.
{{with "output"}}{{printf "%q" .}}{{end}}
A with action using dot.
{{with $x := "output" | printf "%q"}}{{$x}}{{end}}
A with action that creates and uses a variable.
{{with $x := "output"}}{{printf "%q" $x}}{{end}}
A with action that uses the variable in another action.
{{with $x := "output"}}{{$x | printf "%q"}}{{end}}
The same, but pipelined.
Functions
During execution functions are found in two function maps: first in the
template, then in the global function map. By default, no functions are defined
in the template but the Funcs method can be used to add them.
Predefined global functions are named as follows.
and
Returns the boolean AND of its arguments by returning the
first empty argument or the last argument. That is,
"and x y" behaves as "if x then y else x."
Evaluation proceeds through the arguments left to right
and returns when the result is determined.
call
Returns the result of calling the first argument, which
must be a function, with the remaining arguments as parameters.
Thus "call .X.Y 1 2" is, in Go notation, dot.X.Y(1, 2) where
Y is a func-valued field, map entry, or the like.
The first argument must be the result of an evaluation
that yields a value of function type (as distinct from
a predefined function such as print). The function must
return either one or two result values, the second of which
is of type error. If the arguments don't match the function
or the returned error value is non-nil, execution stops.
html
Returns the escaped HTML equivalent of the textual
representation of its arguments. This function is unavailable
in html/template, with a few exceptions.
index
Returns the result of indexing its first argument by the
following arguments. Thus "index x 1 2 3" is, in Go syntax,
x[1][2][3]. Each indexed item must be a map, slice, or array.
slice
slice returns the result of slicing its first argument by the
remaining arguments. Thus "slice x 1 2" is, in Go syntax, x[1:2],
while "slice x" is x[:], "slice x 1" is x[1:], and "slice x 1 2 3"
is x[1:2:3]. The first argument must be a string, slice, or array.
js
Returns the escaped JavaScript equivalent of the textual
representation of its arguments.
len
Returns the integer length of its argument.
not
Returns the boolean negation of its single argument.
or
Returns the boolean OR of its arguments by returning the
first non-empty argument or the last argument, that is,
"or x y" behaves as "if x then x else y".
Evaluation proceeds through the arguments left to right
and returns when the result is determined.
print
An alias for fmt.Sprint
printf
An alias for fmt.Sprintf
println
An alias for fmt.Sprintln
urlquery
Returns the escaped value of the textual representation of
its arguments in a form suitable for embedding in a URL query.
This function is unavailable in html/template, with a few
exceptions.
The boolean functions take any zero value to be false and a non-zero
value to be true.
There is also a set of binary comparison operators defined as
functions:
eq
Returns the boolean truth of arg1 == arg2
ne
Returns the boolean truth of arg1 != arg2
lt
Returns the boolean truth of arg1 < arg2
le
Returns the boolean truth of arg1 <= arg2
gt
Returns the boolean truth of arg1 > arg2
ge
Returns the boolean truth of arg1 >= arg2
For simpler multi-way equality tests, eq (only) accepts two or more
arguments and compares the second and subsequent to the first,
returning in effect
arg1==arg2 || arg1==arg3 || arg1==arg4 ...
(Unlike with || in Go, however, eq is a function call and all the
arguments will be evaluated.)
The comparison functions work on any values whose type Go defines as
comparable. For basic types such as integers, the rules are relaxed:
size and exact type are ignored, so any integer value, signed or unsigned,
may be compared with any other integer value. (The arithmetic value is compared,
not the bit pattern, so all negative integers are less than all unsigned integers.)
However, as usual, one may not compare an int with a float32 and so on.
Associated templates
Each template is named by a string specified when it is created. Also, each
template is associated with zero or more other templates that it may invoke by
name; such associations are transitive and form a name space of templates.
A template may use a template invocation to instantiate another associated
template; see the explanation of the "template" action above. The name must be
that of a template associated with the template that contains the invocation.
Nested template definitions
When parsing a template, another template may be defined and associated with the
template being parsed. Template definitions must appear at the top level of the
template, much like global variables in a Go program.
The syntax of such definitions is to surround each template declaration with a
"define" and "end" action.
The define action names the template being created by providing a string
constant. Here is a simple example:
{{define "T1"}}ONE{{end}}
{{define "T2"}}TWO{{end}}
{{define "T3"}}{{template "T1"}} {{template "T2"}}{{end}}
{{template "T3"}}
This defines two templates, T1 and T2, and a third T3 that invokes the other two
when it is executed. Finally it invokes T3. If executed this template will
produce the text
ONE TWO
By construction, a template may reside in only one association. If it's
necessary to have a template addressable from multiple associations, the
template definition must be parsed multiple times to create distinct *Template
values, or must be copied with [Template.Clone] or [Template.AddParseTree].
Parse may be called multiple times to assemble the various associated templates;
see [ParseFiles], [ParseGlob], [Template.ParseFiles] and [Template.ParseGlob]
for simple ways to parse related templates stored in files.
A template may be executed directly or through [Template.ExecuteTemplate], which executes
an associated template identified by name. To invoke our example above, we
might write,
err := tmpl.Execute(os.Stdout, "no data needed")
if err != nil {
log.Fatalf("execution failed: %s", err)
}
or to invoke a particular template explicitly by name,
err := tmpl.ExecuteTemplate(os.Stdout, "T2", "no data needed")
if err != nil {
log.Fatalf("execution failed: %s", err)
}
*/
package template
File diff suppressed because it is too large Load Diff
+783
View File
@@ -0,0 +1,783 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package template
import (
"errors"
"fmt"
"io"
"net/url"
"reflect"
"strings"
"sync"
"unicode"
"unicode/utf8"
)
// FuncMap is the type of the map defining the mapping from names to functions.
// Each function must have either a single return value, or two return values of
// which the second has type error. In that case, if the second (error)
// return value evaluates to non-nil during execution, execution terminates and
// Execute returns that error.
//
// Errors returned by Execute wrap the underlying error; call [errors.As] to
// unwrap them.
//
// When template execution invokes a function with an argument list, that list
// must be assignable to the function's parameter types. Functions meant to
// apply to arguments of arbitrary type can use parameters of type interface{} or
// of type [reflect.Value]. Similarly, functions meant to return a result of arbitrary
// type can return interface{} or [reflect.Value].
type FuncMap map[string]any
// builtins returns the FuncMap.
// It is not a global variable so the linker can dead code eliminate
// more when this isn't called. See golang.org/issue/36021.
// TODO: revert this back to a global map once golang.org/issue/2559 is fixed.
func builtins() FuncMap {
return FuncMap{
"and": and,
"call": emptyCall,
"html": HTMLEscaper,
"index": index,
"slice": slice,
"js": JSEscaper,
"len": length,
"not": not,
"or": or,
"print": fmt.Sprint,
"printf": fmt.Sprintf,
"println": fmt.Sprintln,
"urlquery": URLQueryEscaper,
// Comparisons
"eq": eq, // ==
"ge": ge, // >=
"gt": gt, // >
"le": le, // <=
"lt": lt, // <
"ne": ne, // !=
}
}
var builtinFuncsOnce struct {
sync.Once
v map[string]reflect.Value
}
// builtinFuncsOnce lazily computes & caches the builtinFuncs map.
// TODO: revert this back to a global map once golang.org/issue/2559 is fixed.
func builtinFuncs() map[string]reflect.Value {
builtinFuncsOnce.Do(func() {
builtinFuncsOnce.v = createValueFuncs(builtins())
})
return builtinFuncsOnce.v
}
// createValueFuncs turns a FuncMap into a map[string]reflect.Value
func createValueFuncs(funcMap FuncMap) map[string]reflect.Value {
m := make(map[string]reflect.Value)
addValueFuncs(m, funcMap)
return m
}
// addValueFuncs adds to values the functions in funcs, converting them to reflect.Values.
func addValueFuncs(out map[string]reflect.Value, in FuncMap) {
for name, fn := range in {
if !goodName(name) {
panic(fmt.Errorf("function name %q is not a valid identifier", name))
}
v := reflect.ValueOf(fn)
if v.Kind() != reflect.Func {
panic("value for " + name + " not a function")
}
if err := goodFunc(name, v.Type()); err != nil {
panic(err)
}
out[name] = v
}
}
// addFuncs adds to values the functions in funcs. It does no checking of the input -
// call addValueFuncs first.
func addFuncs(out, in FuncMap) {
for name, fn := range in {
out[name] = fn
}
}
// goodFunc reports whether the function or method has the right result signature.
func goodFunc(name string, typ reflect.Type) error {
// We allow functions with 1 result or 2 results where the second is an error.
switch numOut := typ.NumOut(); {
case numOut == 1:
return nil
case numOut == 2 && typ.Out(1) == errorType:
return nil
case numOut == 2:
return fmt.Errorf("invalid function signature for %s: second return value should be error; is %s", name, typ.Out(1))
default:
return fmt.Errorf("function %s has %d return values; should be 1 or 2", name, typ.NumOut())
}
}
// goodName reports whether the function name is a valid identifier.
func goodName(name string) bool {
if name == "" {
return false
}
for i, r := range name {
switch {
case r == '_':
case i == 0 && !unicode.IsLetter(r):
return false
case !unicode.IsLetter(r) && !unicode.IsDigit(r):
return false
}
}
return true
}
// findFunction looks for a function in the template, and global map.
func findFunction(name string, tmpl *Template) (v reflect.Value, isBuiltin, ok bool) {
if tmpl != nil && tmpl.common != nil {
tmpl.muFuncs.RLock()
defer tmpl.muFuncs.RUnlock()
if fn := tmpl.execFuncs[name]; fn.IsValid() {
return fn, false, true
}
}
if fn := builtinFuncs()[name]; fn.IsValid() {
return fn, true, true
}
return reflect.Value{}, false, false
}
// prepareArg checks if value can be used as an argument of type argType, and
// converts an invalid value to appropriate zero if possible.
func prepareArg(value reflect.Value, argType reflect.Type) (reflect.Value, error) {
if !value.IsValid() {
if !canBeNil(argType) {
return reflect.Value{}, fmt.Errorf("value is nil; should be of type %s", argType)
}
value = reflect.Zero(argType)
}
if value.Type().AssignableTo(argType) {
return value, nil
}
if intLike(value.Kind()) && intLike(argType.Kind()) && value.Type().ConvertibleTo(argType) {
value = value.Convert(argType)
return value, nil
}
return reflect.Value{}, fmt.Errorf("value has type %s; should be %s", value.Type(), argType)
}
func intLike(typ reflect.Kind) bool {
switch typ {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return true
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
return true
}
return false
}
// indexArg checks if a reflect.Value can be used as an index, and converts it to int if possible.
func indexArg(index reflect.Value, cap int) (int, error) {
var x int64
switch index.Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
x = index.Int()
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
x = int64(index.Uint())
case reflect.Invalid:
return 0, fmt.Errorf("cannot index slice/array with nil")
default:
return 0, fmt.Errorf("cannot index slice/array with type %s", index.Type())
}
if x < 0 || int(x) < 0 || int(x) > cap {
return 0, fmt.Errorf("index out of range: %d", x)
}
return int(x), nil
}
// Indexing.
// index returns the result of indexing its first argument by the following
// arguments. Thus "index x 1 2 3" is, in Go syntax, x[1][2][3]. Each
// indexed item must be a map, slice, or array.
func index(item reflect.Value, indexes ...reflect.Value) (reflect.Value, error) {
item = indirectInterface(item)
if !item.IsValid() {
return reflect.Value{}, fmt.Errorf("index of untyped nil")
}
for _, index := range indexes {
index = indirectInterface(index)
var isNil bool
if item, isNil = indirect(item); isNil {
return reflect.Value{}, fmt.Errorf("index of nil pointer")
}
switch item.Kind() {
case reflect.Array, reflect.Slice, reflect.String:
x, err := indexArg(index, item.Len())
if err != nil {
return reflect.Value{}, err
}
item = item.Index(x)
case reflect.Map:
index, err := prepareArg(index, item.Type().Key())
if err != nil {
return reflect.Value{}, err
}
if x := item.MapIndex(index); x.IsValid() {
item = x
} else {
item = reflect.Zero(item.Type().Elem())
}
case reflect.Invalid:
// the loop holds invariant: item.IsValid()
panic("unreachable")
default:
return reflect.Value{}, fmt.Errorf("can't index item of type %s", item.Type())
}
}
return item, nil
}
// Slicing.
// slice returns the result of slicing its first argument by the remaining
// arguments. Thus "slice x 1 2" is, in Go syntax, x[1:2], while "slice x"
// is x[:], "slice x 1" is x[1:], and "slice x 1 2 3" is x[1:2:3]. The first
// argument must be a string, slice, or array.
func slice(item reflect.Value, indexes ...reflect.Value) (reflect.Value, error) {
item = indirectInterface(item)
if !item.IsValid() {
return reflect.Value{}, fmt.Errorf("slice of untyped nil")
}
if len(indexes) > 3 {
return reflect.Value{}, fmt.Errorf("too many slice indexes: %d", len(indexes))
}
var cap int
switch item.Kind() {
case reflect.String:
if len(indexes) == 3 {
return reflect.Value{}, fmt.Errorf("cannot 3-index slice a string")
}
cap = item.Len()
case reflect.Array, reflect.Slice:
cap = item.Cap()
default:
return reflect.Value{}, fmt.Errorf("can't slice item of type %s", item.Type())
}
// set default values for cases item[:], item[i:].
idx := [3]int{0, item.Len()}
for i, index := range indexes {
x, err := indexArg(index, cap)
if err != nil {
return reflect.Value{}, err
}
idx[i] = x
}
// given item[i:j], make sure i <= j.
if idx[0] > idx[1] {
return reflect.Value{}, fmt.Errorf("invalid slice index: %d > %d", idx[0], idx[1])
}
if len(indexes) < 3 {
return item.Slice(idx[0], idx[1]), nil
}
// given item[i:j:k], make sure i <= j <= k.
if idx[1] > idx[2] {
return reflect.Value{}, fmt.Errorf("invalid slice index: %d > %d", idx[1], idx[2])
}
return item.Slice3(idx[0], idx[1], idx[2]), nil
}
// Length
// length returns the length of the item, with an error if it has no defined length.
func length(item reflect.Value) (int, error) {
item, isNil := indirect(item)
if isNil {
return 0, fmt.Errorf("len of nil pointer")
}
switch item.Kind() {
case reflect.Array, reflect.Chan, reflect.Map, reflect.Slice, reflect.String:
return item.Len(), nil
}
return 0, fmt.Errorf("len of type %s", item.Type())
}
// Function invocation
func emptyCall(fn reflect.Value, args ...reflect.Value) reflect.Value {
panic("unreachable") // implemented as a special case in evalCall
}
// call returns the result of evaluating the first argument as a function.
// The function must return 1 result, or 2 results, the second of which is an error.
func call(name string, fn reflect.Value, args ...reflect.Value) (reflect.Value, error) {
fn = indirectInterface(fn)
if !fn.IsValid() {
return reflect.Value{}, fmt.Errorf("call of nil")
}
typ := fn.Type()
if typ.Kind() != reflect.Func {
return reflect.Value{}, fmt.Errorf("non-function %s of type %s", name, typ)
}
if err := goodFunc(name, typ); err != nil {
return reflect.Value{}, err
}
numIn := typ.NumIn()
var dddType reflect.Type
if typ.IsVariadic() {
if len(args) < numIn-1 {
return reflect.Value{}, fmt.Errorf("wrong number of args for %s: got %d want at least %d", name, len(args), numIn-1)
}
dddType = typ.In(numIn - 1).Elem()
} else {
if len(args) != numIn {
return reflect.Value{}, fmt.Errorf("wrong number of args for %s: got %d want %d", name, len(args), numIn)
}
}
argv := make([]reflect.Value, len(args))
for i, arg := range args {
arg = indirectInterface(arg)
// Compute the expected type. Clumsy because of variadics.
argType := dddType
if !typ.IsVariadic() || i < numIn-1 {
argType = typ.In(i)
}
var err error
if argv[i], err = prepareArg(arg, argType); err != nil {
return reflect.Value{}, fmt.Errorf("arg %d: %w", i, err)
}
}
return safeCall(fn, argv)
}
// safeCall runs fun.Call(args), and returns the resulting value and error, if
// any. If the call panics, the panic value is returned as an error.
func safeCall(fun reflect.Value, args []reflect.Value) (val reflect.Value, err error) {
defer func() {
if r := recover(); r != nil {
if e, ok := r.(error); ok {
err = e
} else {
err = fmt.Errorf("%v", r)
}
}
}()
ret := fun.Call(args)
if len(ret) == 2 && !ret[1].IsNil() {
return ret[0], ret[1].Interface().(error)
}
return ret[0], nil
}
// Boolean logic.
func truth(arg reflect.Value) bool {
t, _ := isTrue(indirectInterface(arg))
return t
}
// and computes the Boolean AND of its arguments, returning
// the first false argument it encounters, or the last argument.
func and(arg0 reflect.Value, args ...reflect.Value) reflect.Value {
panic("unreachable") // implemented as a special case in evalCall
}
// or computes the Boolean OR of its arguments, returning
// the first true argument it encounters, or the last argument.
func or(arg0 reflect.Value, args ...reflect.Value) reflect.Value {
panic("unreachable") // implemented as a special case in evalCall
}
// not returns the Boolean negation of its argument.
func not(arg reflect.Value) bool {
return !truth(arg)
}
// Comparison.
// TODO: Perhaps allow comparison between signed and unsigned integers.
var (
errBadComparisonType = errors.New("invalid type for comparison")
errNoComparison = errors.New("missing argument for comparison")
)
type kind int
const (
invalidKind kind = iota
boolKind
complexKind
intKind
floatKind
stringKind
uintKind
)
func basicKind(v reflect.Value) (kind, error) {
switch v.Kind() {
case reflect.Bool:
return boolKind, nil
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return intKind, nil
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
return uintKind, nil
case reflect.Float32, reflect.Float64:
return floatKind, nil
case reflect.Complex64, reflect.Complex128:
return complexKind, nil
case reflect.String:
return stringKind, nil
}
return invalidKind, errBadComparisonType
}
// isNil returns true if v is the zero reflect.Value, or nil of its type.
func isNil(v reflect.Value) bool {
if !v.IsValid() {
return true
}
switch v.Kind() {
case reflect.Chan, reflect.Func, reflect.Interface, reflect.Map, reflect.Pointer, reflect.Slice:
return v.IsNil()
}
return false
}
// canCompare reports whether v1 and v2 are both the same kind, or one is nil.
// Called only when dealing with nillable types, or there's about to be an error.
func canCompare(v1, v2 reflect.Value) bool {
k1 := v1.Kind()
k2 := v2.Kind()
if k1 == k2 {
return true
}
// We know the type can be compared to nil.
return k1 == reflect.Invalid || k2 == reflect.Invalid
}
// eq evaluates the comparison a == b || a == c || ...
func eq(arg1 reflect.Value, arg2 ...reflect.Value) (bool, error) {
arg1 = indirectInterface(arg1)
if len(arg2) == 0 {
return false, errNoComparison
}
k1, _ := basicKind(arg1)
for _, arg := range arg2 {
arg = indirectInterface(arg)
k2, _ := basicKind(arg)
truth := false
if k1 != k2 {
// Special case: Can compare integer values regardless of type's sign.
switch {
case k1 == intKind && k2 == uintKind:
truth = arg1.Int() >= 0 && uint64(arg1.Int()) == arg.Uint()
case k1 == uintKind && k2 == intKind:
truth = arg.Int() >= 0 && arg1.Uint() == uint64(arg.Int())
default:
if arg1.IsValid() && arg.IsValid() {
return false, fmt.Errorf("incompatible types for comparison: %v and %v", arg1.Type(), arg.Type())
}
}
} else {
switch k1 {
case boolKind:
truth = arg1.Bool() == arg.Bool()
case complexKind:
truth = arg1.Complex() == arg.Complex()
case floatKind:
truth = arg1.Float() == arg.Float()
case intKind:
truth = arg1.Int() == arg.Int()
case stringKind:
truth = arg1.String() == arg.String()
case uintKind:
truth = arg1.Uint() == arg.Uint()
default:
if !canCompare(arg1, arg) {
return false, fmt.Errorf("non-comparable types %s: %v, %s: %v", arg1, arg1.Type(), arg.Type(), arg)
}
if isNil(arg1) || isNil(arg) {
truth = isNil(arg) == isNil(arg1)
} else {
if !arg.Type().Comparable() {
return false, fmt.Errorf("non-comparable type %s: %v", arg, arg.Type())
}
truth = arg1.Interface() == arg.Interface()
}
}
}
if truth {
return true, nil
}
}
return false, nil
}
// ne evaluates the comparison a != b.
func ne(arg1, arg2 reflect.Value) (bool, error) {
// != is the inverse of ==.
equal, err := eq(arg1, arg2)
return !equal, err
}
// lt evaluates the comparison a < b.
func lt(arg1, arg2 reflect.Value) (bool, error) {
arg1 = indirectInterface(arg1)
k1, err := basicKind(arg1)
if err != nil {
return false, err
}
arg2 = indirectInterface(arg2)
k2, err := basicKind(arg2)
if err != nil {
return false, err
}
truth := false
if k1 != k2 {
// Special case: Can compare integer values regardless of type's sign.
switch {
case k1 == intKind && k2 == uintKind:
truth = arg1.Int() < 0 || uint64(arg1.Int()) < arg2.Uint()
case k1 == uintKind && k2 == intKind:
truth = arg2.Int() >= 0 && arg1.Uint() < uint64(arg2.Int())
default:
return false, fmt.Errorf("incompatible types for comparison: %v and %v", arg1.Type(), arg2.Type())
}
} else {
switch k1 {
case boolKind, complexKind:
return false, errBadComparisonType
case floatKind:
truth = arg1.Float() < arg2.Float()
case intKind:
truth = arg1.Int() < arg2.Int()
case stringKind:
truth = arg1.String() < arg2.String()
case uintKind:
truth = arg1.Uint() < arg2.Uint()
default:
panic("invalid kind")
}
}
return truth, nil
}
// le evaluates the comparison <= b.
func le(arg1, arg2 reflect.Value) (bool, error) {
// <= is < or ==.
lessThan, err := lt(arg1, arg2)
if lessThan || err != nil {
return lessThan, err
}
return eq(arg1, arg2)
}
// gt evaluates the comparison a > b.
func gt(arg1, arg2 reflect.Value) (bool, error) {
// > is the inverse of <=.
lessOrEqual, err := le(arg1, arg2)
if err != nil {
return false, err
}
return !lessOrEqual, nil
}
// ge evaluates the comparison a >= b.
func ge(arg1, arg2 reflect.Value) (bool, error) {
// >= is the inverse of <.
lessThan, err := lt(arg1, arg2)
if err != nil {
return false, err
}
return !lessThan, nil
}
// HTML escaping.
var (
htmlQuot = []byte("&#34;") // shorter than "&quot;"
htmlApos = []byte("&#39;") // shorter than "&apos;" and apos was not in HTML until HTML5
htmlAmp = []byte("&amp;")
htmlLt = []byte("&lt;")
htmlGt = []byte("&gt;")
htmlNull = []byte("\uFFFD")
)
// HTMLEscape writes to w the escaped HTML equivalent of the plain text data b.
func HTMLEscape(w io.Writer, b []byte) {
last := 0
for i, c := range b {
var html []byte
switch c {
case '\000':
html = htmlNull
case '"':
html = htmlQuot
case '\'':
html = htmlApos
case '&':
html = htmlAmp
case '<':
html = htmlLt
case '>':
html = htmlGt
default:
continue
}
w.Write(b[last:i])
w.Write(html)
last = i + 1
}
w.Write(b[last:])
}
// HTMLEscapeString returns the escaped HTML equivalent of the plain text data s.
func HTMLEscapeString(s string) string {
// Avoid allocation if we can.
if !strings.ContainsAny(s, "'\"&<>\000") {
return s
}
var b strings.Builder
HTMLEscape(&b, []byte(s))
return b.String()
}
// HTMLEscaper returns the escaped HTML equivalent of the textual
// representation of its arguments.
func HTMLEscaper(args ...any) string {
return HTMLEscapeString(evalArgs(args))
}
// JavaScript escaping.
var (
jsLowUni = []byte(`\u00`)
hex = []byte("0123456789ABCDEF")
jsBackslash = []byte(`\\`)
jsApos = []byte(`\'`)
jsQuot = []byte(`\"`)
jsLt = []byte(`\u003C`)
jsGt = []byte(`\u003E`)
jsAmp = []byte(`\u0026`)
jsEq = []byte(`\u003D`)
)
// JSEscape writes to w the escaped JavaScript equivalent of the plain text data b.
func JSEscape(w io.Writer, b []byte) {
last := 0
for i := 0; i < len(b); i++ {
c := b[i]
if !jsIsSpecial(rune(c)) {
// fast path: nothing to do
continue
}
w.Write(b[last:i])
if c < utf8.RuneSelf {
// Quotes, slashes and angle brackets get quoted.
// Control characters get written as \u00XX.
switch c {
case '\\':
w.Write(jsBackslash)
case '\'':
w.Write(jsApos)
case '"':
w.Write(jsQuot)
case '<':
w.Write(jsLt)
case '>':
w.Write(jsGt)
case '&':
w.Write(jsAmp)
case '=':
w.Write(jsEq)
default:
w.Write(jsLowUni)
t, b := c>>4, c&0x0f
w.Write(hex[t : t+1])
w.Write(hex[b : b+1])
}
} else {
// Unicode rune.
r, size := utf8.DecodeRune(b[i:])
if unicode.IsPrint(r) {
w.Write(b[i : i+size])
} else {
fmt.Fprintf(w, "\\u%04X", r)
}
i += size - 1
}
last = i + 1
}
w.Write(b[last:])
}
// JSEscapeString returns the escaped JavaScript equivalent of the plain text data s.
func JSEscapeString(s string) string {
// Avoid allocation if we can.
if strings.IndexFunc(s, jsIsSpecial) < 0 {
return s
}
var b strings.Builder
JSEscape(&b, []byte(s))
return b.String()
}
func jsIsSpecial(r rune) bool {
switch r {
case '\\', '\'', '"', '<', '>', '&', '=':
return true
}
return r < ' ' || utf8.RuneSelf <= r
}
// JSEscaper returns the escaped JavaScript equivalent of the textual
// representation of its arguments.
func JSEscaper(args ...any) string {
return JSEscapeString(evalArgs(args))
}
// URLQueryEscaper returns the escaped value of the textual representation of
// its arguments in a form suitable for embedding in a URL query.
func URLQueryEscaper(args ...any) string {
return url.QueryEscape(evalArgs(args))
}
// evalArgs formats the list of arguments into a string. It is therefore equivalent to
//
// fmt.Sprint(args...)
//
// except that each argument is indirected (if a pointer), as required,
// using the same rules as the default string evaluation during template
// execution.
func evalArgs(args []any) string {
ok := false
var s string
// Fast path for simple common case.
if len(args) == 1 {
s, ok = args[0].(string)
}
if !ok {
for i, arg := range args {
a, ok := printableValue(reflect.ValueOf(arg))
if ok {
args[i] = a
} // else let fmt do its thing
}
s = fmt.Sprint(args...)
}
return s
}
@@ -0,0 +1,154 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package fmtsort provides a general stable ordering mechanism
// for maps, on behalf of the fmt and text/template packages.
// It is not guaranteed to be efficient and works only for types
// that are valid map keys.
package fmtsort
import (
"cmp"
"reflect"
"slices"
)
// Note: Throughout this package we avoid calling reflect.Value.Interface as
// it is not always legal to do so and it's easier to avoid the issue than to face it.
// SortedMap is a slice of KeyValue pairs that simplifies sorting
// and iterating over map entries.
//
// Each KeyValue pair contains a map key and its corresponding value.
type SortedMap []KeyValue
// KeyValue holds a single key and value pair found in a map.
type KeyValue struct {
Key, Value reflect.Value
}
// Sort accepts a map and returns a SortedMap that has the same keys and
// values but in a stable sorted order according to the keys, modulo issues
// raised by unorderable key values such as NaNs.
//
// The ordering rules are more general than with Go's < operator:
//
// - when applicable, nil compares low
// - ints, floats, and strings order by <
// - NaN compares less than non-NaN floats
// - bool compares false before true
// - complex compares real, then imag
// - pointers compare by machine address
// - channel values compare by machine address
// - structs compare each field in turn
// - arrays compare each element in turn.
// Otherwise identical arrays compare by length.
// - interface values compare first by reflect.Type describing the concrete type
// and then by concrete value as described in the previous rules.
func Sort(mapValue reflect.Value) SortedMap {
if mapValue.Type().Kind() != reflect.Map {
return nil
}
// Note: this code is arranged to not panic even in the presence
// of a concurrent map update. The runtime is responsible for
// yelling loudly if that happens. See issue 33275.
n := mapValue.Len()
sorted := make(SortedMap, 0, n)
iter := mapValue.MapRange()
for iter.Next() {
sorted = append(sorted, KeyValue{iter.Key(), iter.Value()})
}
slices.SortStableFunc(sorted, func(a, b KeyValue) int {
return compare(a.Key, b.Key)
})
return sorted
}
// compare compares two values of the same type. It returns -1, 0, 1
// according to whether a > b (1), a == b (0), or a < b (-1).
// If the types differ, it returns -1.
// See the comment on Sort for the comparison rules.
func compare(aVal, bVal reflect.Value) int {
aType, bType := aVal.Type(), bVal.Type()
if aType != bType {
return -1 // No good answer possible, but don't return 0: they're not equal.
}
switch aVal.Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return cmp.Compare(aVal.Int(), bVal.Int())
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
return cmp.Compare(aVal.Uint(), bVal.Uint())
case reflect.String:
return cmp.Compare(aVal.String(), bVal.String())
case reflect.Float32, reflect.Float64:
return cmp.Compare(aVal.Float(), bVal.Float())
case reflect.Complex64, reflect.Complex128:
a, b := aVal.Complex(), bVal.Complex()
if c := cmp.Compare(real(a), real(b)); c != 0 {
return c
}
return cmp.Compare(imag(a), imag(b))
case reflect.Bool:
a, b := aVal.Bool(), bVal.Bool()
switch {
case a == b:
return 0
case a:
return 1
default:
return -1
}
case reflect.Pointer, reflect.UnsafePointer:
return cmp.Compare(aVal.Pointer(), bVal.Pointer())
case reflect.Chan:
if c, ok := nilCompare(aVal, bVal); ok {
return c
}
return cmp.Compare(aVal.Pointer(), bVal.Pointer())
case reflect.Struct:
for i := 0; i < aVal.NumField(); i++ {
if c := compare(aVal.Field(i), bVal.Field(i)); c != 0 {
return c
}
}
return 0
case reflect.Array:
for i := 0; i < aVal.Len(); i++ {
if c := compare(aVal.Index(i), bVal.Index(i)); c != 0 {
return c
}
}
return 0
case reflect.Interface:
if c, ok := nilCompare(aVal, bVal); ok {
return c
}
c := compare(reflect.ValueOf(aVal.Elem().Type()), reflect.ValueOf(bVal.Elem().Type()))
if c != 0 {
return c
}
return compare(aVal.Elem(), bVal.Elem())
default:
// Certain types cannot appear as keys (maps, funcs, slices), but be explicit.
panic("bad type in compare: " + aType.String())
}
}
// nilCompare checks whether either value is nil. If not, the boolean is false.
// If either value is nil, the boolean is true and the integer is the comparison
// value. The comparison is defined to be 0 if both are nil, otherwise the one
// nil value compares low. Both arguments must represent a chan, func,
// interface, map, pointer, or slice.
func nilCompare(aVal, bVal reflect.Value) (int, bool) {
if aVal.IsNil() {
if bVal.IsNil() {
return 0, true
}
return -1, true
}
if bVal.IsNil() {
return 1, true
}
return 0, false
}
+72
View File
@@ -0,0 +1,72 @@
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// This file contains the code to handle template options.
package template
import "strings"
// missingKeyAction defines how to respond to indexing a map with a key that is not present.
type missingKeyAction int
const (
mapInvalid missingKeyAction = iota // Return an invalid reflect.Value.
mapZeroValue // Return the zero value for the map element.
mapError // Error out
)
type option struct {
missingKey missingKeyAction
}
// Option sets options for the template. Options are described by
// strings, either a simple string or "key=value". There can be at
// most one equals sign in an option string. If the option string
// is unrecognized or otherwise invalid, Option panics.
//
// Known options:
//
// missingkey: Control the behavior during execution if a map is
// indexed with a key that is not present in the map.
//
// "missingkey=default" or "missingkey=invalid"
// The default behavior: Do nothing and continue execution.
// If printed, the result of the index operation is the string
// "<no value>".
// "missingkey=zero"
// The operation returns the zero value for the map type's element.
// "missingkey=error"
// Execution stops immediately with an error.
func (t *Template) Option(opt ...string) *Template {
t.init()
for _, s := range opt {
t.setOption(s)
}
return t
}
func (t *Template) setOption(opt string) {
if opt == "" {
panic("empty option string")
}
// key=value
if key, value, ok := strings.Cut(opt, "="); ok {
switch key {
case "missingkey":
switch value {
case "invalid", "default":
t.option.missingKey = mapInvalid
return
case "zero":
t.option.missingKey = mapZeroValue
return
case "error":
t.option.missingKey = mapError
return
}
}
}
panic("unrecognized option: " + opt)
}
+236
View File
@@ -0,0 +1,236 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package template
import (
"maps"
"reflect"
"sync"
"text/template/parse"
)
// common holds the information shared by related templates.
type common struct {
tmpl map[string]*Template // Map from name to defined templates.
muTmpl sync.RWMutex // protects tmpl
option option
// We use two maps, one for parsing and one for execution.
// This separation makes the API cleaner since it doesn't
// expose reflection to the client.
muFuncs sync.RWMutex // protects parseFuncs and execFuncs
parseFuncs FuncMap
execFuncs map[string]reflect.Value
}
// Template is the representation of a parsed template. The *parse.Tree
// field is exported only for use by [html/template] and should be treated
// as unexported by all other clients.
type Template struct {
name string
*parse.Tree
*common
leftDelim string
rightDelim string
}
// New allocates a new, undefined template with the given name.
func New(name string) *Template {
t := &Template{
name: name,
}
t.init()
return t
}
// Name returns the name of the template.
func (t *Template) Name() string {
return t.name
}
// New allocates a new, undefined template associated with the given one and with the same
// delimiters. The association, which is transitive, allows one template to
// invoke another with a {{template}} action.
//
// Because associated templates share underlying data, template construction
// cannot be done safely in parallel. Once the templates are constructed, they
// can be executed in parallel.
func (t *Template) New(name string) *Template {
t.init()
nt := &Template{
name: name,
common: t.common,
leftDelim: t.leftDelim,
rightDelim: t.rightDelim,
}
return nt
}
// init guarantees that t has a valid common structure.
func (t *Template) init() {
if t.common == nil {
c := new(common)
c.tmpl = make(map[string]*Template)
c.parseFuncs = make(FuncMap)
c.execFuncs = make(map[string]reflect.Value)
t.common = c
}
}
// Clone returns a duplicate of the template, including all associated
// templates. The actual representation is not copied, but the name space of
// associated templates is, so further calls to [Template.Parse] in the copy will add
// templates to the copy but not to the original. Clone can be used to prepare
// common templates and use them with variant definitions for other templates
// by adding the variants after the clone is made.
func (t *Template) Clone() (*Template, error) {
nt := t.copy(nil)
nt.init()
if t.common == nil {
return nt, nil
}
nt.option = t.option
t.muTmpl.RLock()
defer t.muTmpl.RUnlock()
for k, v := range t.tmpl {
if k == t.name {
nt.tmpl[t.name] = nt
continue
}
// The associated templates share nt's common structure.
tmpl := v.copy(nt.common)
nt.tmpl[k] = tmpl
}
t.muFuncs.RLock()
defer t.muFuncs.RUnlock()
maps.Copy(nt.parseFuncs, t.parseFuncs)
maps.Copy(nt.execFuncs, t.execFuncs)
return nt, nil
}
// copy returns a shallow copy of t, with common set to the argument.
func (t *Template) copy(c *common) *Template {
return &Template{
name: t.name,
Tree: t.Tree,
common: c,
leftDelim: t.leftDelim,
rightDelim: t.rightDelim,
}
}
// AddParseTree associates the argument parse tree with the template t, giving
// it the specified name. If the template has not been defined, this tree becomes
// its definition. If it has been defined and already has that name, the existing
// definition is replaced; otherwise a new template is created, defined, and returned.
func (t *Template) AddParseTree(name string, tree *parse.Tree) (*Template, error) {
t.init()
t.muTmpl.Lock()
defer t.muTmpl.Unlock()
nt := t
if name != t.name {
nt = t.New(name)
}
// Even if nt == t, we need to install it in the common.tmpl map.
if t.associate(nt, tree) || nt.Tree == nil {
nt.Tree = tree
}
return nt, nil
}
// Templates returns a slice of defined templates associated with t.
func (t *Template) Templates() []*Template {
if t.common == nil {
return nil
}
// Return a slice so we don't expose the map.
t.muTmpl.RLock()
defer t.muTmpl.RUnlock()
m := make([]*Template, 0, len(t.tmpl))
for _, v := range t.tmpl {
m = append(m, v)
}
return m
}
// Delims sets the action delimiters to the specified strings, to be used in
// subsequent calls to [Template.Parse], [Template.ParseFiles], or [Template.ParseGlob]. Nested template
// definitions will inherit the settings. An empty delimiter stands for the
// corresponding default: {{ or }}.
// The return value is the template, so calls can be chained.
func (t *Template) Delims(left, right string) *Template {
t.init()
t.leftDelim = left
t.rightDelim = right
return t
}
// Funcs adds the elements of the argument map to the template's function map.
// It must be called before the template is parsed.
// It panics if a value in the map is not a function with appropriate return
// type or if the name cannot be used syntactically as a function in a template.
// It is legal to overwrite elements of the map. The return value is the template,
// so calls can be chained.
func (t *Template) Funcs(funcMap FuncMap) *Template {
t.init()
t.muFuncs.Lock()
defer t.muFuncs.Unlock()
addValueFuncs(t.execFuncs, funcMap)
addFuncs(t.parseFuncs, funcMap)
return t
}
// Lookup returns the template with the given name that is associated with t.
// It returns nil if there is no such template or the template has no definition.
func (t *Template) Lookup(name string) *Template {
if t.common == nil {
return nil
}
t.muTmpl.RLock()
defer t.muTmpl.RUnlock()
return t.tmpl[name]
}
// Parse parses text as a template body for t.
// Named template definitions ({{define ...}} or {{block ...}} statements) in text
// define additional templates associated with t and are removed from the
// definition of t itself.
//
// Templates can be redefined in successive calls to Parse.
// A template definition with a body containing only white space and comments
// is considered empty and will not replace an existing template's body.
// This allows using Parse to add new named template definitions without
// overwriting the main template body.
func (t *Template) Parse(text string) (*Template, error) {
t.init()
t.muFuncs.RLock()
trees, err := parse.Parse(t.name, text, t.leftDelim, t.rightDelim, t.parseFuncs, builtins())
t.muFuncs.RUnlock()
if err != nil {
return nil, err
}
// Add the newly parsed trees, including the one for t, into our common structure.
for name, tree := range trees {
if _, err := t.AddParseTree(name, tree); err != nil {
return nil, err
}
}
return t, nil
}
// associate installs the new template into the group of templates associated
// with t. The two are already known to share the common structure.
// The boolean return value reports whether to store this tree as t.Tree.
func (t *Template) associate(new *Template, tree *parse.Tree) bool {
if new.common != t.common {
panic("internal error: associate not common")
}
if old := t.tmpl[new.name]; old != nil && parse.IsEmptyTree(tree.Root) && old.Tree != nil {
// If a template by that name exists,
// don't replace it with an empty template.
return false
}
t.tmpl[new.name] = new
return true
}
+8 -1
View File
@@ -3,7 +3,14 @@ package topdown
import (
"bytes"
"strings"
"text/template"
// A method-less copy of text/template (see internal/methodlesstemplate). Rego values
// decode to map[string]any/[]any/scalars, which have no methods, so eliding
// method calls is a no-op here; it keeps text/template's evalField
// MethodByName off the reachable graph, which otherwise disables the Go
// linker's method-level dead-code elimination binary-wide (golang/go#72895,
// #7903).
template "github.com/open-policy-agent/opa/internal/methodlesstemplate"
"github.com/open-policy-agent/opa/v1/ast"
"github.com/open-policy-agent/opa/v1/topdown/builtins"
+45
View File
@@ -0,0 +1,45 @@
package topdown
import (
"go/parser"
"go/token"
"os"
"strings"
"testing"
)
// TestNoStdlibTextTemplateImport guards the linker dead-code-elimination fix
// (#7903): render_template must use the method-less template package, never
// stdlib text/template or html/template. Their evalField reaches
// reflect.Value.MethodByName, whose reachability disables method-level DCE for
// the whole binary of every OPA embedder. Reintroducing the import silently
// regresses that. Scans every non-test file in the package so the guard holds
// even if the import moves to another file or template.go is renamed.
func TestNoStdlibTextTemplateImport(t *testing.T) {
entries, err := os.ReadDir(".")
if err != nil {
t.Fatal(err)
}
fset := token.NewFileSet()
scanned := 0
for _, e := range entries {
name := e.Name()
if e.IsDir() || !strings.HasSuffix(name, ".go") || strings.HasSuffix(name, "_test.go") {
continue
}
f, err := parser.ParseFile(fset, name, nil, parser.ImportsOnly)
if err != nil {
t.Fatalf("parse %s: %v", name, err)
}
scanned++
for _, imp := range f.Imports {
switch strings.Trim(imp.Path.Value, `"`) {
case "text/template", "html/template":
t.Errorf("%s imports %s — reintroduces the linker DCE-defeat trigger", name, imp.Path.Value)
}
}
}
if scanned == 0 {
t.Fatal("scanned no package files; test ran from the wrong directory")
}
}