* wasm: emit ABI version as global
This takes inspiration from the proxy-spec (Envoy's Wasm support).
There, it's recorded in an exported function's name. However, it's
been included like that in the spec because it's the least common
denominator among the different languages (potentially) used to
implement proxy-spec. We've got a pretty good grip on our generated
Wasm code, so we do what's noted in proxy-spec as "ideally, we'd do
xyz instead".
However, our ABI version is a simple integer, no semver.
Ref: https://github.com/proxy-wasm/spec/tree/master/abi-versions/vNEXT#proxy_abi_version_x_y_z
* ast.CapabilitiesForThisVersion: include WasmABIVersions
Extending the ast.Capabilities like this is somewhat unsatisfying -- the Wasm ABI has little to do with the ast package. However, moving Capabilities outside of ast in a way that's not introducing import cycles and is backwards-compatible proved to be quite an effort; so let's go with "simple" here.
* capatibilities.json: ensure it is generated with ABI versions
The build tag `generate` is what `go generate` would set, too. We're losing
that in the main.go -> gen-run-go.sh indirection, so we've got to set it
ourselves.
* ci: fix npm-opa-wasm e2e test
The CI build uses a version of OPA built in a previous step -- with the Wasm SDK _disabled_.
To still build Wasm modules, we thus fix the call to use the capabilities.json file from master,
which corresponds to the capabilities of a build of OPA with Wasm SDK enabled.
* docs/content/wasm.md: mention abi version, change headers
There is only one `#` header in a markdown document, so this fixes
that by adding a few `#`. I haven't added it everywhere below
`# Compiling`, but I think the structure is OK now.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
The codecov check is not providing any value. Over time we have had to
tweak the check parameters because of false-positives. As well, the
service is often unavailable when clicking through from GitHub. The
final straw was today when the check stopped showing inside of PRs. At
this point everyone is ignoring it.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This attempts to give us some more safety for WASM-related changes.
We don't want to break the SDK!
It's introducing some coordination efforts in the future: when a WASM-
related change is supposed to be merged that requires SDKs to change,
we'll have to merge that into npm-opa-wasm first, before the PR can go
in. However, it's better to have this sort of breaking change smoke
test.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Since this more than doubled build times, run only on pushes to master/release branches and not on each PR.
Signed-off-by: Anders Eknert <anders@eknert.com>
Rather than have the tests run as part of the normal go `make test`
target we add a new one specific to them, and a github action for PR's
to match.
This helps keep reduce the response time for PR's waiting on golang
unit tests significantly.
Signed-off-by: Patrick East <east.patrick@gmail.com>
The post tag workflow used the set-env command to
set the TAG_NAME env variable. The set-env command is
now disabled due to a security vulnerability in the GitHub Actions
runner that can allow environment variable and path injection in
workflows that log untrusted data to STDOUT. For more information see:
https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/
This change updates the workflow to remove
usage of the set-env command to use environment files
instead.
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
The "post-merge" flow would trigger whenever a commit is pushed, which
includes when we auto-generate the wasm binaries. If (for whatever
reason) that generate step was non-deterministic it would potentially
mean the CI looping and adding commits until we manually intervene.
This changes to prevent the step from being able to commit on top of
another auto-generated commit.
Signed-off-by: Patrick East <east.patrick@gmail.com>
Rather than force PR's to include the generated wasm binaries we can
accept changes which require regenerating them. The post merge
workflow will now generate them and commit+push the new binaries.
The PR check for generated changes includes new functionality to
exclude files, the first ones being the wasm generated ones.
Signed-off-by: Patrick East <east.patrick@gmail.com>
Previously we had some required configuration for the Github Actions,
and forks of OPA would need to set them _and_ have the underlying
infra configured (eg, docker registries, s3 bucket, etc).
Now it will check if the secrets are set, and if any required ones
are missing it will skip the steps.
This significantly lowers the bar for OPA forks to be able to run the
normal action workflows without getting errors. The only lost
functionality is primarily around publishing release assets, which
is not required for dev forks, and other forks can opt int to pieces
they care about (eg, only want to publish docker images and no
s3 release assets).
Signed-off-by: Patrick East <east.patrick@gmail.com>
We previously supported overriding via an environment variable, but
this meant for anyone who wanted to run their own telemetry endpoint
they would _always_ have to specify it while running their OPA's.
This change allows for someone to build OPA and encode the custom
url as the default. Ex:
```
make build TELEMETRY_URL=http://localhost:9876/custom/
```
Signed-off-by: Patrick East <east.patrick@gmail.com>
Update the conditional syntax to be valid and switch to a different
slack action helper that provides a better end result.
Signed-off-by: Patrick East <east.patrick@gmail.com>
We are using a 3rd party action to simplify this. It appears to be
relatively well used, and the code looked pretty safe. It only has
access to the slack webhook secret, which is itself restricted in
permissions, so the risk is minimal.
It is configured to post a message for jobs that fail to the OPA
slack in the #development channel.
Signed-off-by: Patrick East <east.patrick@gmail.com>
This adds the tooling from:
https://github.com/tsandall/fuzz-opa
plus some new helper scripts and make targets to use it as a pass/fail
CI step.
The script will (as of now) run the fuzzing for an hour and raise an
error if any crashers were found. We can adjust the timing as needed,
this initial setting is pretty arbitrary.
Signed-off-by: Patrick East <east.patrick@gmail.com>
We will run the golang race detector nightly (to start with.. we'll
adjust the workflow as needed).
One thing to note is that currently cgo is required for the race
detector, so we have to enable it when running this make target.
Fixes: #2388
Signed-off-by: Patrick East <east.patrick@gmail.com>
We were only doing it on the PR workflows, but we should have one
uploaded on master after merges to keep an up to date baseline.
Signed-off-by: Patrick East <east.patrick@gmail.com>
This includes some refactors to the build steps. High level items:
* Add variables for DOCKER_IMAGE, S3_RELEASE_BUCKET to allow for forks
of OPA to re-use the GitHub actions with their own s3 buckets and
docker orgs/image names.
* Unify the release build steps to use `make release` and the binaries
being located under `_release/$(VERSION)`. All CI targets now rely
on binaries being in that `RELEASE_DIR`, including image building
steps The `make build` target is unaffected.
* Add a wrapper to allow the CI to run the various golang target
stages separately, but sharing the same docker configuration.
* Conditionally specify `-it` for docker run commands based on whether
A tty is available.
* Added scripts to automate drafting a release with binary assets vi
the `hub` CLI.
* The release process triggered on a tag being pushed will now use the
same binaries from `make release` for the docker images as well as
the ones attached to the release (which are available under
https://openpolicyagent.org/downloads/).
The actions themselves are split into 3 workflows:
pull-request.yaml:
Triggers on pull requests. This will run all the normal tests/checks
as before on Travis, however they are now split into separate jobs.
In addition to what was done on Travis we will now have Codecov
results included.
post-merge.yaml:
Triggers after a change is pushed to master. This will run tests and
build+publish the `edge` and `dev` artifacts to dockerhub and s3.
post-tag.yaml:
Triggers after a tag has been pushed. Similar to post-merge.yaml it
will run tests and build+publish release artifacts (for the tagged
version). It will also create a draft release on GitHub with the
same artifacts and notes from the CHANGELOG.md. If a release already
exists it will be updated to include the assets, however the release
notes will _not_ be added.
The RELEASE.md steps have been updated and include notes on the new
steps.
Signed-off-by: Patrick East <east.patrick@gmail.com>
Stephan Renatus