All packages, except for `cmd` and `internal`, have been moved into a new `v1` root package.
Old packages are kept for backwards-compatibility reasons. All contained code is replaced with simple type aliases and proxy functions to `v1` implementations.
Old packages default to the Rego v0 syntax, new `v1` packages default to the Rego v1 syntax.
Signed-off-by: Johan Fylling <johan.dev@fylling.se>
Decision logs had previously been configured to hide the value of the
Authorization header, as that is considered sensitive information.
However, there are cases when additional headers are provided that
contain sensitive information, such as the X-AMZ-SECURITY-TOKEN header.
This PR creates an internal map of headers that should be masked, which
can be expanded if additional headers are required. It then loops over
the headers in a request, and performs a lookup on the internal map
to see if any of them match those that should be masked. If so, it
replaces their values with "REDACTED". An existing test was added to
check both the header keys that should be masked, as well as a key
that should not.
Additional work, out of scope for this PR, would be to open a config
setting that would allow users to pass in a list of headers that should
be masked.
Fixes: #5848
Signed-off-by: Colin Lacy <colinjlacy@gmail.com>
This change addresses solutions 2) and 3) of the related issue #5553.
It mainly starts using the (now exposed) `Config.AuthPlugin()` function
of the `rest` package in the `download.OCIDownloader`. This allows it
to use any `HTTPAuthPlugin` that is defined in the `Config.Credentials`
section and makes it much more consistent with behavior of the
`download.Downloader` and potential other uses of the rest package.
Fixes#5553
Signed-off-by: DerGut <jannik.steinmann@gmx.de>
This commit updates the client debug log to include
the full HTTP response in case of non-200 status codes.
Recording the response in the logs can help to provide
more information to debug error scenarios.
Fixes: #2961
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
Initial support for #4518.
Configuration uses the 'services' config for registries, via the "type: oci" field.
Bundles configured to pull from that service will then use OCI.
```
services:
ghcr-registry:
url: https://ghcr.io
type: oci
bundles:
authz:
service: ghcr-registry
resource: ghcr.io/${ORGANIZATION}/${REPOSITORY}:${TAG}
persist: true
polling:
min_delay_seconds: 60
max_delay_seconds: 120
persistence_directory: ${PERSISTENCE_PATH}
```
Service credentials are supported: if you want to pull from a private registry,
use
```
services:
ghcr-registry:
url: https://ghcr.io
type: oci
credentials:
bearer:
token: ${GH_PAT}
```
If no `persistence_directory` is configured, the data is stored in a directory under /tmp.
See docs/devel/OCI.md for manual steps to test this feature with some
OCI registry (like ghcr.io).
Signed-off-by: carabasdaniel <dani@aserto.com>
* chore: improves auth plugin resolution.
Currently when aiming to use a Plugin in credentials section, if the plugin is known then it will be resolved, if it isn't, it will be passed to the supported credentials and tried to be cast as HTTPAuthPlugin which ends up in a casting issue without further feedback on what was the plugin string.
Signed-off-by: José Carlos Chávez <jcchavezs@gmail.com>
This removes the GetFields function from the logger interface, as mentioned in #4114.
GetFields used to be called in one place, creating a new logger using fields from an
http client afaict. I am not sure if my changes have the desired effect in that case,
or how this was desired to work - since the fields of the client are always changing
when making requests.
Fixes#4114.
Signed-off-by: viovanov <vlad@aserto.com>
This commit adds an HTTP auth plugin that fetches bearer access tokens using managed identities for Azure resources. This plugin will complement the existing AWS and GCP auth plugins.
Signed-off-by: David Lu <david.scowluga@gmail.com>
This commit fixes the console loggers so that messages are emitted
regardless of the debug log level. The problem was that in 3fcc875 we
updated the plugins to use a console logger obtained from the plugin
manager as opposed to a global logger instantiated in the plugins
package--the console logger obtained from the plugin manager was
instantiated in the runtime package by calling
logging.NewStandardLogger. Unfortunately, logging.NewStandardLogger
does not create a new logger--it returns the global logrus
logger.
This commit fixes the issue by deprecating logging.NewStandardLogger
and introducing two new functions in the logging package:
* logging.Get() - this replaces the old logging.NewStandardLogger
function--this function should be called to obtain the debug logger
used throughout OPA.
* logging.New() - this actually returns a new logger that can be
configured independently from the debug logger used throughout
OPA.
The runtime and sdk packages have been updated to call logging.New()
to obtain console loggers and the rest of the codebase has been
updated to call logging.Get() in place of logging.NewStandardLogger().
Fixes#3654
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This change allows users to specify a certificate for the services
that implement the bundle, status etc. APIs. This cert will be
used to create the root CA pool.
Fixes: #1954
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
golint is deprecated. The author of the code no longer supports the
codebase. golangci-lint is faster than golint, and is in use by other
opa repositories (e.g. Gatekeeper).
This commit changes tools.go to reference golangci (so it ends up in
vendor) and modifies check-lint to use golangci instead.
Breaking API Changes:
- plugins/rest/rest.go: Fix typo "AllowInsureTLS" -> "AllowInsecureTLS"
- storage/errors.go: Removed unused IndexingNotSupportedErr
Signed-off-by: Will Beason <willbeason@google.com>
This commit moves the logging interface and implementations out of the
sdk package into the logging package.
This commit also updates the status and decision log plugins to use a
logger obtained from the plugin manager instead of going to the global
console logger in the plugins package. The latter change will be
important for SDK consumers. This change is backwards incompatible but
it's unlikely that anyone is relying on that export. The test for
console logger independence has also been moved into the plugins
package (from the status package.)
Fixes#3275
Co-authored-by: Torin Sandall <torinsandall@gmail.com>
Co-authored-by: Anders Eknert <anders@eknert.com>
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Signed-off-by: Anders Eknert <anders@eknert.com>
Earlier the downloader package only supported the http
short polling technique where the client sends periodic
requests to the server to fetch bundles. A drawback of this
method is that a low polling frequency could add unnecessary
burden on the server and network.
This commit adds support for http long polling which helps
to minimize server/network resource usage and also reduces
the delay in delivery of updates to the client.
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
Refactor logging to allow providing custom logging implementations to plugin
manager. This should allow us to keep logging as it is when running OPA as a
server, while injecting noop-loggers or custom, provided loggers for SDK client
implementations.
Fixes#3180
Signed-off-by: Anders Eknert <anders@eknert.com>
Allow OPA to issue JWT's which it uses to authenticate a configured
OAuth2 client, as described in RFC7523. This replaces the client_secret
as the actual credential and allows for either using an entirely new
grant type called "JWT bearer", or using the previously supported
client_credentials grant type, only with the client_secret replaced
by a signed JWT. This change covers both scenarios described in
RFC7523.
Other changes made to accomodate this feature:
- Add `private_key` attribute to keys struct to allow for both public and
private keys to be stored there.
- Refactored the keys configuration struct and logic to its
own package no longer coupled to bundles.
Closes#3055
Signed-off-by: Anders Eknert <anders@eknert.com>
Plugins that implement the HTTPAuthPlugin can be used with a new
credentials options under services:
```
services:
my_service:
credentials:
plugin: my_plugin
plugins:
my_plugin: {}
```
Fixes#2758
Signed-off-by: Grant Shively <gshively@godaddy.com>
Adds support for fetching access and identity tokens from a GCP
metadata server. Identity tokens are used to authenticate to third
party applications running behind Google authentication proxies
such as containers deployed to Google's Cloud Run.
Access tokens are used to authenticate to first party GCP services
such as Google Cloud Storage.
Signed-off-by: Kelsey Hightower <kelsey.hightower@gmail.com>
Allows OPA to obtain OAuth2 access tokens and present them as bearer tokens for authentication at remote endpoints.
Fixes#1205
Signed-off-by: Anders Eknert <anders.eknert@bisnode.com>
This commit adds a new configurable timeout to the Services
config to set the amount of time to wait for the server's
response headers. With this change, the client will no longer
wait indefinitely for the HTTP request to complete.
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
Earlier with discovery enabled, there was no protection against accidental
changes to the discovery service. This change prevents the discovery service
from being modified by checking it's config in the service bundle.
Fixes#2058
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
Earlier with discovery enabled updates to the 'services' configuration
was not allowed to protect against accidental changes to the discovery service
itself. Since adding new services could be useful, this change allows modifications
to the 'services' configuration. The only exception is that the service used to download
the discovery bundle cannot be modified.
Fixes#2058
Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
The HTTP client we were using for bundle downloads was being loaded
once and then re-used. This was problematic for users that configure
cert files and then have them change.
This updates the client wrapper in OPA to re-create the underlying
client for each request.
Fixes: #1898
Signed-off-by: Patrick East <east.patrick@gmail.com>
This can be overridden by custom header configuration for services,
or requests but will default to something like:
`Open Policy Agent/<version> (<os>, <arch>)`
This is set on all outbound requests for status, decision logs, bundle
and discovery downloads, http requests from policies, etc.
Signed-off-by: Patrick East <east.patrick@gmail.com>
When instantiating a http.Transport struct without providing non-zero
values for some of its fields, it will leak connections.
The docs hint at this (https://golang.org/pkg/net/http/#Transport):
By default, Transport caches connections for future re-use. This may
leave many open connections when accessing many hosts. This behavior
can be managed using Transport's CloseIdleConnections method and the
MaxIdleConnsPerHost and DisableKeepAlives fields.
See this issue for details: https://github.com/golang/go/issues/19620
I have not observed any leaks in the wild for this, but I'm also
not using this plugin. I came to fix this because I had recently
introduced a bug in the server tests -- they were altering the
settings for the global http.DefaultTransport when my intention
was to create a copy. Digging in while fixing that revealed this
other location. (The fix for the server tests is in the next commit).
Signed-off-by: Stephan Renatus <srenatus@chef.io>
* Fix typo in run short description
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
* Add support for specifying services as object
Previously the services configuration had to be specified as an array,
for example:
services:
- name: foo
url: https://example.com
In some cases, it's easier to structure the configuration as an object.
Specifically, the Helm package manager does not allow you to override
values nested under arrays. These changes allow the services
configuration to be structured as an object:
services:
foo:
url: https://example.com
Ref: https://github.com/helm/helm/issues/1987
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Adds support for usings client certificates to authenticate against
service endpoints. It also enables users to allow insecure TLS
connections to service endpoints.
Fixes#684
Signed-off-by: Kim Christensen <kimworking@gmail.com>
Previously, policies and data had to be pushed into OPA via the REST API
or loaded via command line arguments at startup.
With these changes, OPA can now be configured to pull down bundles of
policy and data from remote HTTP servers. When a bundle is downloaded
successfully, the policies and data are loaded out of the bundle file
and inserted into storage.
Johan Fylling