This uses Trivy to scan both the Git repo, as well as the generated
container image for vulnerabilities. It leverages Trivy's GitHub action
[1]. Currently, it's set to alert on `CRITICAL` and `HIGH`
vulnerabilities only.
[1] https://github.com/aquasecurity/trivy-action
This adds a section in the developer docs, as well as comments on the GitHub workflows.
Signed-off-by: Juan Antonio Osorio <juan.osoriorobles@eu.equinix.com>
To be brought back eventually; for now, it is just randomly failing.
We haven't been touching the Wasm interface in a while, so this test
isn't urgently needed these days.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
* builtin_metadata: add v0.41.0 manually
We'll figure out how to do this in the release process, but for now, this
is enough to not have the builtin_metadata.json change appear in each PR.
* ci: pin last release of npm-opa-wasm
This is temporary; I don't want to deal with this right now.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
* topdown: fix TestRego: run for all go versions, excluding the x509 error
That error has a different message on go1.16.
The previous attempt to exclude them from running caused _all tests_
to not be run.
* topdown_test/TestTopDownQueryCancellationEvery: up wait time for macos flakey tests
We've often seen this fail with "0 notes". Waiting for 10x the time
we previous waited for still seems to do the trick but should
hopefully remove the amount of failures we see in CE because of
slow macos runners.
* ci: don't run wasm build again in compat builds
The build is docker-based, and doesn't differ at all if run from a
different version of golang. So instead of re-building it in the
separate matrix jobs, we'll use the artifacts downloaded from the
artifact build job.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
* build/policies: format using 0.39.0
* workflow/pull-request: use edge opa for rego PR checks
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
No change to go.mod's `go` stanza, so no changes in code compatibility.
However, it's used for building our docker images and release
binaries, and for fuzz testing in our nightly workflow.
Some test-related changes with the dns lookup built-in function's
error handling; and the hardcoded signature. Running
go test ./topdown -run TestTopdownJWTEncodeSignECWithSeedReturnsSameSignature -count 10000
makes me believe that for whatever reason the signature changed,
it's at least stable.
topdown/http_test: Test-only change to accomodate this change in Go (https://go.dev/doc/go1.18):
Certificate.Verify now uses platform APIs to verify certificate
validity on macOS and iOS when it is called with a nil
VerifyOpts.Roots or when using the root pool returned from
SystemCertPool.
We're keeping the old message for go <= 1.17; in a silly-simple way.
Also:
* ci: build and test two old golang version on macos|linux
We'll drop golang 1.15, keep one unsupported version (1.16).
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Fixes#4226.
Also
* adds a YAML test to ensure that this works fine end-to-end.
* ci(pull-request): show input on failure
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
* Makefile: temporary disable "delete" for edge bucket sync
This is a bandaid to be reverted when we've fixed the IAM policy
protecting the bucket to allow for deletions.
* workflow: re-enable token in checkout
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
* build: don't push arm64 binaries to edge releases
They're not built for a release, but we'd still be publishing
them to the edge S3 bucket post-merge.
* build: silence 'aws s3 sync' output
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This change adds linux/arm64 binaries to the release. It also publishes an arm64
container image for all variants (standard, debug, rootless, static) and releases
(dev, edge, latest).
The build and push process uses buildx in order to push the individual
images by digest (i.e. untagged) and reference them in a single, tagged manifest
list. This avoids cluttering Docker Hub's tag list with `<tag>-<arch>` tags.
Fixes#2233
Signed-off-by: Nick Graef <1031317+ngraef@users.noreply.github.com>
Also included some improvements to the Rego checks:
* Pass GITHUB_TOKEN to policy to not exceed API quota
* Ensure required attributes included in integration
* Ensure commited .json files are valid JSON
Signed-off-by: Anders Eknert <anders@eknert.com>
Also:
* Add some links to Kubernetes authorization item
* Add SPIFFE/SPIRE blog
* Extend Rego tests to verify added/modified YAML files as valid
The last point was intended to be for the integrations.yaml file
only, but thinking more about it made sense not to limit the check
to a single file.
Signed-off-by: Anders Eknert <anders@eknert.com>
Since both contributors and reviewers (i.e. me!) seem
to easily miss the correct location of the logo for a new
integration - add checks that will fail the PR when this
happens.
This is admittedly mostly for fun, but I figured it would
be pretty cool to explore whether we could integrate Rego
policies into our own build pipeline. There are definitely
more things to explore using the GitHub API as a datasource
for build pipeline policies, but this is at least a start.
Signed-off-by: Anders Eknert <anders@eknert.com>
Somewhat experimental, but now that pretty much all new macs
run with the ARM64 architecture it would be nice to add it as
a target to our releases. Since there is currently no runner
for GitHub Actions (https://github.com/actions/runner/issues/805)
we can't yet run the binary smoke test for this architecture,
but I'm tracking the issue and hoping that can be resolved soon.
Feel free to dismiss this if you think this should wait until later.
Signed-off-by: Anders Eknert <anders@eknert.com>
It seldomly matters for PRs, since only a tiny subset of them alters
dependencies. Having the check run in nightlies, where a failure does
not block a PR, but we still notice it through the notifications,
seems like a good trade-off.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Since the golang stdlib function doesn't do any caching, we add the result
to the BuiltinContext.Cache so it's cached, and consistent, within a single
policy evaluation.
There is no decision made here about using netgo or netcgo: we're following
suit wrt how golang expects you to do it: From my understanding, using the
OS means for DNS resolution is the preferred way: it gives you per-host
caching, and it allows the user to affect how DNS resolution works in many
ways.
This means the same logic that applies to all other places where we resolve
domain names into addresses (notably `http.send`) applies to this built-in,
too.
Also:
* workflow/pull_request: don't fail-fast for matrix jobs
Even if one platform fails it would be interesting to see what happens
on the others.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This way we do not have to run the entire test and benchmark suite
that takes about 25m in total.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
* Do not run ci-release-tag target--the tests have already been run
pre-merge and post-merge so there is little reason to run them again
post-tag. The only thing this would do is find non-deterministic
test failures--which begs the question: what do we do with the
release? We already run tests pre-merge, post-merge, and nightly so
it's unlikely that post-tag will help improve quality.
* Use the RELEASE_DIR from the makefile for the `hub release` asset
parameter rather than assuming the TAG <=> RELEASE_DIR (this is not
always true if tagging an arbitrary commit.) This enables us to cut
release candidates without commiting changes to the repo.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This installs gotip (the lastest build) and uses its (beta) fuzzing feature in
the nightly tests.
We can remove the previous setup at a later date.
The "seed corpus" was converted from the previous fuzzer's using this script:
package main
import (
"bytes"
"fmt"
"io/ioutil"
"log"
"path/filepath"
)
const oldCorpusDir = "build/fuzzer/corpus/"
const newCorpusDir = "ast/testdata/fuzz/FuzzParseStatementsAndCompileModules"
func main() {
files, err := ioutil.ReadDir(oldCorpusDir)
if err != nil {
log.Fatal(err)
}
for _, f := range files {
c, err := ioutil.ReadFile(filepath.Join(oldCorpusDir, f.Name()))
if err != nil {
log.Fatal(err)
}
buf := bytes.Buffer{}
buf.WriteString("go test fuzz v1\n")
fmt.Fprintf(&buf, "string(%q)\n", string(c))
err = ioutil.WriteFile(filepath.Join(newCorpusDir, f.Name()), buf.Bytes(), 0644)
if err != nil {
log.Fatal(err)
}
}
}
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This extra check is meant to catch go module proxy checksum mismatches,
like the one we've released 0.32.1 to fix, earlier.
It causes the go mod tooling to fetch all modules from their external sources,
most likely all github references, and compares the contents' checksums with
what we have in go.sum. It deliberately bypasses the "sumdb" service that is
part of the golang infrastructure.
The event of a mismatch would happen if a git tag was published, and later
changed, and the golang infrastructure's module proxy (and sumdb service)
had picked up the first tag. This is rather unlikely, and this test is thus a bit
over-cautious. The idea is that if it becomes invisible, it's fine to keep, and
gives us a bit of extra safety. However, if it becomes annoying (it's a giant
network dependency in our CI runs), it's not critical enough to be kept and
is OK to disable again.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
This splits off the generate call, so that we can have the wasm bits (and capabilities.json) from the PR used in the PR checks.
The Perf check still is the one taking longest, even with the added matrix build job and the jobs depending on it. The test matrix job was split off to run those in parallel, the binary smoke tests for example can already proceed.
To simplify things, we're relying on the setup-go action for both the linux and the darwin unit tests. (We could also use it for the builds of linux and windows binaries... but there, I'm more concerned about a clean build env and reproducibility.)
Fixes#3176.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
We only check the build, not the tests.
And we only check the latest release of the 1.15 and 1.16 series.
since 1.17 is what we build and test with anyways.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
dependabot[bot]