This change adds linux/arm64 binaries to the release. It also publishes an arm64
container image for all variants (standard, debug, rootless, static) and releases
(dev, edge, latest).
The build and push process uses buildx in order to push the individual
images by digest (i.e. untagged) and reference them in a single, tagged manifest
list. This avoids cluttering Docker Hub's tag list with `<tag>-<arch>` tags.
Fixes#2233
Signed-off-by: Nick Graef <1031317+ngraef@users.noreply.github.com>
* build: add static (wasm-disabled) linux build
Fixes#3499.
Also:
* build: deprecate 'release' and 'release-local' targets that aren't used in
our build anymore, and will go away eventually.
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
* build: WASM_ENABLED=1 for all platforms, bump go to 1.16.3
Notes:
- If there are other users of the 'build-windows' make target they would
likely be annoyed by the change that's now apt-get'ting packages
- We could build a builder image instead of installing the package every
time.
- ci-go-*: run as root now, so we're able to install the packages for
windows.
- tests: skip tests that depend on not being run as root when root. The
change to ci-go-* makes that necessary; the impact is rather limited
right now. We can reconsider if there are more tests depending on not
being run as root.
- build: add '-buildmode=exe' to GOFLAGS
Primarily for the windows build, but I don't think it should be wrong
for the others either:
https://github.com/golang/go/issues/40795
See https://golang.org/cmd/go/#hdr-Build_modes:
> -buildmode=exe
> Build the listed main packages and everything they import into
> executables. Packages not named main are ignored.
- go: fix version as 1.16.3 (not 1.16)
We'd rather keep this an exact match.
- build: update go module related env vars
With 1.16, https://blog.golang.org/go116-module-changes,
> The go command now builds packages in module-aware mode by default.
Also, since we've added the `go 1.15` directive to go.mod, we can drop
all -mod=vendor flags, https://golang.org/ref/mod#go-mod-file-go,
> At go 1.14 or higher, automatic vendoring may be enabled. If the file
> vendor/modules.txt is present and consistent with go.mod, there is no
> need to explicitly use the -mod=vendor flag.
- build: override docker id/gid in 'image' target, to keep existing
behaviour.
* workflow: use binaries built before, remove workaround
split linux and windows to not wait for the windows build to finish
before starting the npm-opa-wasm tests.
* wasm-sdk: show where to get binaries, don't panic
Fixes#3264.
* Makefile: deprecate old targets, introduce new ones
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
We originally set the rpath to be relative to the $ORIGIN which
apparently changes if the working directory is overriden. This change
just fixes the rpath to contain /usr/lib/opa and then updates the
Dockerfile to copy the libwasmer.so file into this new directory.
Fixes#2974
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This commit changes a few build settings required for the wasm runtime
integration.
Since wasmer ships as a shared library, it must be distributed with
OPA. Rather than require all users obtain the wasmer shared library,
we continue to publish statically linked OPA executables like we
always have. For the time being, these statically linked executables
will NOT have wasm runtime support available. If users want to
interact with wasm compiled policies, they will either have to do so
via Docker or build OPA themselves.
As part of this change, the Docker images have been updated to be from
gcr.io/distroless/cc (since libc and libgcc are required now) instead
of scratch.
Finally, the build process now produces an extra "opa_docker_$GOARCH"
executable that is used in the Docker image. This executable has the
runpath set so that it can find the wasmer library at runtime.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This includes some refactors to the build steps. High level items:
* Add variables for DOCKER_IMAGE, S3_RELEASE_BUCKET to allow for forks
of OPA to re-use the GitHub actions with their own s3 buckets and
docker orgs/image names.
* Unify the release build steps to use `make release` and the binaries
being located under `_release/$(VERSION)`. All CI targets now rely
on binaries being in that `RELEASE_DIR`, including image building
steps The `make build` target is unaffected.
* Add a wrapper to allow the CI to run the various golang target
stages separately, but sharing the same docker configuration.
* Conditionally specify `-it` for docker run commands based on whether
A tty is available.
* Added scripts to automate drafting a release with binary assets vi
the `hub` CLI.
* The release process triggered on a tag being pushed will now use the
same binaries from `make release` for the docker images as well as
the ones attached to the release (which are available under
https://openpolicyagent.org/downloads/).
The actions themselves are split into 3 workflows:
pull-request.yaml:
Triggers on pull requests. This will run all the normal tests/checks
as before on Travis, however they are now split into separate jobs.
In addition to what was done on Travis we will now have Codecov
results included.
post-merge.yaml:
Triggers after a change is pushed to master. This will run tests and
build+publish the `edge` and `dev` artifacts to dockerhub and s3.
post-tag.yaml:
Triggers after a tag has been pushed. Similar to post-merge.yaml it
will run tests and build+publish release artifacts (for the tagged
version). It will also create a draft release on GitHub with the
same artifacts and notes from the CHANGELOG.md. If a release already
exists it will be updated to include the assets, however the release
notes will _not_ be added.
The RELEASE.md steps have been updated and include notes on the new
steps.
Signed-off-by: Patrick East <east.patrick@gmail.com>
The scratch base image does not contain ca-certificates which meant
that the version check was failing because of TLS errors while
connecting to telemetry.openpolicyagent.org.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Now that we can build with CGO_ENABLED=0 we do not have to use a base
image that provides libc. Remove distroless for the normal and
rootless images and use scratch instead.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This commit combines a few changes to the build. Namely:
1. The travis build no longer works off a dirty working copy. The
.dockerignore file was excluding the docs directory which caused the
working copy to become dirty during the build process. While this
isn't a huge issue it does make it harder to be confident about the
state of the source that Travis binaries are built from. As part of
this change, we remove the builder image in favour of running the
golang image and volume mounting the working copy. This is avoids the
copy that is quite expensive in the OPA repo.
2. In the recent build refactoring, the wasm development workflow was
broken. Changes to the wasm library were not getting picked up
automatically when running the wasm/rego tests. This commit fixes the
makefile so that the wasm libary is rebuilt and the wasm blob is
copied and regenerated each time the wasm/rego tests are run.
Finally, this commit leans into modules a bit more removing the
scheduler test dependency on GOPATH.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
We use the gcr.io/distroless/base:debug tag for the *-debug images of
OPA. At the point where we switched to the multi-stage build we dropped
defining the `VARIANT` argument so it wasn't being picked up correctly.
This lead to us just using gcr.io/distroless/base for the *-debug
image.. which isn't what we wanted.
Signed-off-by: Patrick East <east.patrick@gmail.com>
As discussed in #1639.
Note that .dockerignore had to be changed as to *not* skip what's needed
for the build: vendor, obviously, and .git for the compiled-in version
information.
parts:
- .travis.yml: don't bother about golang anymore
- docs/devel/DEVELOPMENT.md: update
- Makefile: update hint
Signed-off-by: Stephan Renatus <srenatus@chef.io>
Matúš Ferech