Also included some improvements to the Rego checks:
* Pass GITHUB_TOKEN to policy to not exceed API quota
* Ensure required attributes included in integration
* Ensure commited .json files are valid JSON
Signed-off-by: Anders Eknert <anders@eknert.com>
Also:
* Add some links to Kubernetes authorization item
* Add SPIFFE/SPIRE blog
* Extend Rego tests to verify added/modified YAML files as valid
The last point was intended to be for the integrations.yaml file
only, but thinking more about it made sense not to limit the check
to a single file.
Signed-off-by: Anders Eknert <anders@eknert.com>
Since both contributors and reviewers (i.e. me!) seem
to easily miss the correct location of the logo for a new
integration - add checks that will fail the PR when this
happens.
This is admittedly mostly for fun, but I figured it would
be pretty cool to explore whether we could integrate Rego
policies into our own build pipeline. There are definitely
more things to explore using the GitHub API as a datasource
for build pipeline policies, but this is at least a start.
Signed-off-by: Anders Eknert <anders@eknert.com>
Somewhat experimental, but now that pretty much all new macs
run with the ARM64 architecture it would be nice to add it as
a target to our releases. Since there is currently no runner
for GitHub Actions (https://github.com/actions/runner/issues/805)
we can't yet run the binary smoke test for this architecture,
but I'm tracking the issue and hoping that can be resolved soon.
Feel free to dismiss this if you think this should wait until later.
Signed-off-by: Anders Eknert <anders@eknert.com>
It seldomly matters for PRs, since only a tiny subset of them alters
dependencies. Having the check run in nightlies, where a failure does
not block a PR, but we still notice it through the notifications,
seems like a good trade-off.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Since the golang stdlib function doesn't do any caching, we add the result
to the BuiltinContext.Cache so it's cached, and consistent, within a single
policy evaluation.
There is no decision made here about using netgo or netcgo: we're following
suit wrt how golang expects you to do it: From my understanding, using the
OS means for DNS resolution is the preferred way: it gives you per-host
caching, and it allows the user to affect how DNS resolution works in many
ways.
This means the same logic that applies to all other places where we resolve
domain names into addresses (notably `http.send`) applies to this built-in,
too.
Also:
* workflow/pull_request: don't fail-fast for matrix jobs
Even if one platform fails it would be interesting to see what happens
on the others.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This way we do not have to run the entire test and benchmark suite
that takes about 25m in total.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
* Do not run ci-release-tag target--the tests have already been run
pre-merge and post-merge so there is little reason to run them again
post-tag. The only thing this would do is find non-deterministic
test failures--which begs the question: what do we do with the
release? We already run tests pre-merge, post-merge, and nightly so
it's unlikely that post-tag will help improve quality.
* Use the RELEASE_DIR from the makefile for the `hub release` asset
parameter rather than assuming the TAG <=> RELEASE_DIR (this is not
always true if tagging an arbitrary commit.) This enables us to cut
release candidates without commiting changes to the repo.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This installs gotip (the lastest build) and uses its (beta) fuzzing feature in
the nightly tests.
We can remove the previous setup at a later date.
The "seed corpus" was converted from the previous fuzzer's using this script:
package main
import (
"bytes"
"fmt"
"io/ioutil"
"log"
"path/filepath"
)
const oldCorpusDir = "build/fuzzer/corpus/"
const newCorpusDir = "ast/testdata/fuzz/FuzzParseStatementsAndCompileModules"
func main() {
files, err := ioutil.ReadDir(oldCorpusDir)
if err != nil {
log.Fatal(err)
}
for _, f := range files {
c, err := ioutil.ReadFile(filepath.Join(oldCorpusDir, f.Name()))
if err != nil {
log.Fatal(err)
}
buf := bytes.Buffer{}
buf.WriteString("go test fuzz v1\n")
fmt.Fprintf(&buf, "string(%q)\n", string(c))
err = ioutil.WriteFile(filepath.Join(newCorpusDir, f.Name()), buf.Bytes(), 0644)
if err != nil {
log.Fatal(err)
}
}
}
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This extra check is meant to catch go module proxy checksum mismatches,
like the one we've released 0.32.1 to fix, earlier.
It causes the go mod tooling to fetch all modules from their external sources,
most likely all github references, and compares the contents' checksums with
what we have in go.sum. It deliberately bypasses the "sumdb" service that is
part of the golang infrastructure.
The event of a mismatch would happen if a git tag was published, and later
changed, and the golang infrastructure's module proxy (and sumdb service)
had picked up the first tag. This is rather unlikely, and this test is thus a bit
over-cautious. The idea is that if it becomes invisible, it's fine to keep, and
gives us a bit of extra safety. However, if it becomes annoying (it's a giant
network dependency in our CI runs), it's not critical enough to be kept and
is OK to disable again.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This splits off the generate call, so that we can have the wasm bits (and capabilities.json) from the PR used in the PR checks.
The Perf check still is the one taking longest, even with the added matrix build job and the jobs depending on it. The test matrix job was split off to run those in parallel, the binary smoke tests for example can already proceed.
To simplify things, we're relying on the setup-go action for both the linux and the darwin unit tests. (We could also use it for the builds of linux and windows binaries... but there, I'm more concerned about a clean build env and reproducibility.)
Fixes#3176.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
We only check the build, not the tests.
And we only check the latest release of the 1.15 and 1.16 series.
since 1.17 is what we build and test with anyways.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This means that "make lint" will work the same for everyone and the
workflow without additional coordination. Before this PR, it was
possible for maintainers to be on a different version of golangci-lint
than the one used by the GitHub workflow and so "make check" would
provide inconsistent results.
Also add timeout to configuration so we don't time out.`
Signed-off-by: Will Beason <willbeason@google.com>
* build: add static (wasm-disabled) linux build
Fixes#3499.
Also:
* build: deprecate 'release' and 'release-local' targets that aren't used in
our build anymore, and will go away eventually.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
The post-tag workflow was broken in the recent release. This commit adds
the generate step back into the workflow.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
* workflow: add 'quick fuzz' run
This is just a very short (3m) fuzz check, to ensure that we don't
surprise ourselves when merging something that in some weird way
affects the nightly fuzzer run. It's probably unlikely to come up
with interesting results, but that's what the long nightly run is for.
* build/fuzzer: fix go module state
Something went amiss here. It seems like running these two commands
makes the build work again:
$ go mod tidy
$ go get github.com/dvyukov/go-fuzz/go-fuzz-dep
go get: added github.com/dvyukov/go-fuzz v0.0.0-20210429054444-fca39067bc72
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
* workflow: fix codeql go version
* cmd/fmt_test: use ioutil.Discard
I had proposed that in a PR review... not thinking that it meant
we couldn't run tests using Go 1.15. My assumption that using
go 1.15
in go.mod would protect us from that was wrong, apparently.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
* runtime_test: avoid race condition
This had been flagged by our nightly race deteector run. Now, we'll
wait for the server to have stopped before checking its log output.
* plugins: avoid races, bump github.com/sirupsen/logrus
To fix that other one, I've first tried updating logrus (there was a
mention of fixed races in the changelog), but to no avail. Setting up
the hook before any plugin would log from that test resolved the issue.
No harm in updating logrus, though, let's keep that: 1.6.0 -> 1.8.1
* plugins/bundle: fix race
Golang for-range loops need special care when using a reference to the
second variable (v in `for k, v := range m`). We had been copying the
value of m[k], which is a pointer to Status, we had not been -- as was
intended -- copying the values of the struct that the pointer had been
pointing to.
Tests needed to be adapted for this, the s4 update will NOT contain
any bundle-activation-related metrics, as no bundle was activated, and
its status is a fresh copy.
* workflow: add race detector to PR checks
When run from nightly, we use ubuntu-latest; whereas the other checks
in the pull-request workflow use ubuntu-18.04.
I don't think it matters at all for the race detector, since that one
runs only from another docker container, using the golang image.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
* build: WASM_ENABLED=1 for all platforms, bump go to 1.16.3
Notes:
- If there are other users of the 'build-windows' make target they would
likely be annoyed by the change that's now apt-get'ting packages
- We could build a builder image instead of installing the package every
time.
- ci-go-*: run as root now, so we're able to install the packages for
windows.
- tests: skip tests that depend on not being run as root when root. The
change to ci-go-* makes that necessary; the impact is rather limited
right now. We can reconsider if there are more tests depending on not
being run as root.
- build: add '-buildmode=exe' to GOFLAGS
Primarily for the windows build, but I don't think it should be wrong
for the others either:
https://github.com/golang/go/issues/40795
See https://golang.org/cmd/go/#hdr-Build_modes:
> -buildmode=exe
> Build the listed main packages and everything they import into
> executables. Packages not named main are ignored.
- go: fix version as 1.16.3 (not 1.16)
We'd rather keep this an exact match.
- build: update go module related env vars
With 1.16, https://blog.golang.org/go116-module-changes,
> The go command now builds packages in module-aware mode by default.
Also, since we've added the `go 1.15` directive to go.mod, we can drop
all -mod=vendor flags, https://golang.org/ref/mod#go-mod-file-go,
> At go 1.14 or higher, automatic vendoring may be enabled. If the file
> vendor/modules.txt is present and consistent with go.mod, there is no
> need to explicitly use the -mod=vendor flag.
- build: override docker id/gid in 'image' target, to keep existing
behaviour.
* workflow: use binaries built before, remove workaround
split linux and windows to not wait for the windows build to finish
before starting the npm-opa-wasm tests.
* wasm-sdk: show where to get binaries, don't panic
Fixes#3264.
* Makefile: deprecate old targets, introduce new ones
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
* wasm: emit ABI version as global
This takes inspiration from the proxy-spec (Envoy's Wasm support).
There, it's recorded in an exported function's name. However, it's
been included like that in the spec because it's the least common
denominator among the different languages (potentially) used to
implement proxy-spec. We've got a pretty good grip on our generated
Wasm code, so we do what's noted in proxy-spec as "ideally, we'd do
xyz instead".
However, our ABI version is a simple integer, no semver.
Ref: https://github.com/proxy-wasm/spec/tree/master/abi-versions/vNEXT#proxy_abi_version_x_y_z
* ast.CapabilitiesForThisVersion: include WasmABIVersions
Extending the ast.Capabilities like this is somewhat unsatisfying -- the Wasm ABI has little to do with the ast package. However, moving Capabilities outside of ast in a way that's not introducing import cycles and is backwards-compatible proved to be quite an effort; so let's go with "simple" here.
* capatibilities.json: ensure it is generated with ABI versions
The build tag `generate` is what `go generate` would set, too. We're losing
that in the main.go -> gen-run-go.sh indirection, so we've got to set it
ourselves.
* ci: fix npm-opa-wasm e2e test
The CI build uses a version of OPA built in a previous step -- with the Wasm SDK _disabled_.
To still build Wasm modules, we thus fix the call to use the capabilities.json file from master,
which corresponds to the capabilities of a build of OPA with Wasm SDK enabled.
* docs/content/wasm.md: mention abi version, change headers
There is only one `#` header in a markdown document, so this fixes
that by adding a few `#`. I haven't added it everywhere below
`# Compiling`, but I think the structure is OK now.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
The codecov check is not providing any value. Over time we have had to
tweak the check parameters because of false-positives. As well, the
service is often unavailable when clicking through from GitHub. The
final straw was today when the check stopped showing inside of PRs. At
this point everyone is ignoring it.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This attempts to give us some more safety for WASM-related changes.
We don't want to break the SDK!
It's introducing some coordination efforts in the future: when a WASM-
related change is supposed to be merged that requires SDKs to change,
we'll have to merge that into npm-opa-wasm first, before the PR can go
in. However, it's better to have this sort of breaking change smoke
test.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Since this more than doubled build times, run only on pushes to master/release branches and not on each PR.
Signed-off-by: Anders Eknert <anders@eknert.com>
Rather than have the tests run as part of the normal go `make test`
target we add a new one specific to them, and a github action for PR's
to match.
This helps keep reduce the response time for PR's waiting on golang
unit tests significantly.
Signed-off-by: Patrick East <east.patrick@gmail.com>
The post tag workflow used the set-env command to
set the TAG_NAME env variable. The set-env command is
now disabled due to a security vulnerability in the GitHub Actions
runner that can allow environment variable and path injection in
workflows that log untrusted data to STDOUT. For more information see:
https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/
This change updates the workflow to remove
usage of the set-env command to use environment files
instead.
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
The "post-merge" flow would trigger whenever a commit is pushed, which
includes when we auto-generate the wasm binaries. If (for whatever
reason) that generate step was non-deterministic it would potentially
mean the CI looping and adding commits until we manually intervene.
This changes to prevent the step from being able to commit on top of
another auto-generated commit.
Signed-off-by: Patrick East <east.patrick@gmail.com>
Rather than force PR's to include the generated wasm binaries we can
accept changes which require regenerating them. The post merge
workflow will now generate them and commit+push the new binaries.
The PR check for generated changes includes new functionality to
exclude files, the first ones being the wasm generated ones.
Signed-off-by: Patrick East <east.patrick@gmail.com>
Previously we had some required configuration for the Github Actions,
and forks of OPA would need to set them _and_ have the underlying
infra configured (eg, docker registries, s3 bucket, etc).
Now it will check if the secrets are set, and if any required ones
are missing it will skip the steps.
This significantly lowers the bar for OPA forks to be able to run the
normal action workflows without getting errors. The only lost
functionality is primarily around publishing release assets, which
is not required for dev forks, and other forks can opt int to pieces
they care about (eg, only want to publish docker images and no
s3 release assets).
Signed-off-by: Patrick East <east.patrick@gmail.com>
We previously supported overriding via an environment variable, but
this meant for anyone who wanted to run their own telemetry endpoint
they would _always_ have to specify it while running their OPA's.
This change allows for someone to build OPA and encode the custom
url as the default. Ex:
```
make build TELEMETRY_URL=http://localhost:9876/custom/
```
Signed-off-by: Patrick East <east.patrick@gmail.com>
Update the conditional syntax to be valid and switch to a different
slack action helper that provides a better end result.
Signed-off-by: Patrick East <east.patrick@gmail.com>
We are using a 3rd party action to simplify this. It appears to be
relatively well used, and the code looked pretty safe. It only has
access to the slack webhook secret, which is itself restricted in
permissions, so the risk is minimal.
It is configured to post a message for jobs that fail to the OPA
slack in the #development channel.
Signed-off-by: Patrick East <east.patrick@gmail.com>
This adds the tooling from:
https://github.com/tsandall/fuzz-opa
plus some new helper scripts and make targets to use it as a pass/fail
CI step.
The script will (as of now) run the fuzzing for an hour and raise an
error if any crashers were found. We can adjust the timing as needed,
this initial setting is pretty arbitrary.
Signed-off-by: Patrick East <east.patrick@gmail.com>
We will run the golang race detector nightly (to start with.. we'll
adjust the workflow as needed).
One thing to note is that currently cgo is required for the race
detector, so we have to enable it when running this make target.
Fixes: #2388
Signed-off-by: Patrick East <east.patrick@gmail.com>
We were only doing it on the PR workflows, but we should have one
uploaded on master after merges to keep an up to date baseline.
Signed-off-by: Patrick East <east.patrick@gmail.com>
Anders Eknert