Commit Graph

2411 Commits

Author SHA1 Message Date
Torin Sandall 8becfb8b39 Dockerfile: Go back to scratch
Now that we can build with CGO_ENABLED=0 we do not have to use a base
image that provides libc. Remove distroless for the normal and
rootless images and use scratch instead.

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-30 14:53:20 -04:00
Torin Sandall 7db82e7aa3 cmd: Remove support for shared library loading
Shared library loading was removed in v0.14.0. This commit removes the
deprecated code and moves the test coverage for plugin registration
via the runtime package global into the runtime package.

Fixes #2049

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-30 14:53:20 -04:00
Teemu Koponen eacecb01bc wasm: Parse and write JSON unicode strings.
This adds support for both UTF-8 and UTF-16. All the JSON value
strings use internal representation of UTF-8. Unicode validation and
translation to UTF-8 is performed only at the JSON parsing time.
There's no further encoding validation at the writing time but it is
assumed all the string operations maintain the validity of UTF-8
representation.

Fixes #1885

Signed-off-by: Teemu Koponen <koponen@styra.com>
2020-04-30 08:48:17 -04:00
Srestha Srivastava d1b839474c docs: Correct the link to opa bulitins in external data page
Signed-off-by: Srestha Srivastava <sresthasrivastava.ss@gmail.com>
2020-04-29 15:04:36 -07:00
Patrick East 2f283dcc6a format: Preserve "else" block style when possible
We used to always force empty lines between else blocks and their
parent rule. Recently we changed to force them into a compact/inline
style as long as there were no comments between blocks.. now we will
preserve the original style, favoring the compact style.

This should make the change that happened more backwards compatible
and allow policy authors to use either style without the formatter
forcing one over the other.

Signed-off-by: Patrick East <east.patrick@gmail.com>
2020-04-29 13:26:39 -07:00
Patrick East cb11315b27 ast: Set location text for rules when parsing
Previously we didn't have the correct rule location text (for the full
rule, head, value, else's, etc). This changes to set the text and adds
some tests to validate.

This affects the formatter somewhat with how it decides where to place
comments. The current output now more accurately reflects their
placement in the original policies.

There is one little hack to set the else head value locations, there
will need to be some more significant changes to the formatter before
we can change that.

Signed-off-by: Patrick East <east.patrick@gmail.com>
2020-04-29 13:26:39 -07:00
Torin Sandall 30a757b8c2 ast: Fix parser to include body on else if needed
In other places where the body can be implied, the parser includes a
body containing a single "true" expression. Do the same for else.

Fixes #2353

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-29 12:47:14 -07:00
Ashutosh Narkar 5d2b9df39d plugins: Additive updates to services when discovery enabled
Earlier with discovery enabled, there was no protection against accidental
changes to the discovery service. This change prevents the discovery service
from being modified by checking it's config in the service bundle.

Fixes #2058

Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
2020-04-29 06:36:10 -04:00
Patrick East f7747e7826 Update release note for 0.19.2
Signed-off-by: Patrick East <east.patrick@gmail.com>
2020-04-27 15:54:33 -07:00
Torin Sandall ae39df236f runtime: Remove unnecessary info field
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-27 15:26:43 -07:00
Torin Sandall 62f292b6e3 internal: Refactor load/store/compile implementation
This commit refactors the load/store/compile implementation that used
to live inside the runtime package. Specifically:

* Move init-time file loading logic into separate internal package
  (initload) along with store/compile logic. Add tests around
  load/store/compile that don't require the entire Runtime object.
  This also avoids duplication of the "version overwriting" logic.

* Move store/compile calls into the manager. This avoids the need for
  two compile operations on startup.

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-27 15:26:43 -07:00
Torin Sandall c79bdc4b17 server: Fix query error test case to avoid dependency on manager start
This test case does not need to have the manager started and since the
manager now performs at least one read on start, the mock used in the
test case would cause the test to fail.

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-27 15:26:43 -07:00
Torin Sandall f962990758 plugins: Fix race between manager and plugin startup
This change fixes a race condition in the manager that was caused by
registering the storage trigger _after_ the plugins had been
started. The problem was that if the bundle plugin was able to
download and activate before the trigger registration in the manager
went through, the store and the manager would be out-of-sync after
startup. The bundle would activate successfully but the plugin
manager would not see the change. This meant that the server health
check, status plugin, etc. would report successful activation and
clients using either of those APIs for synchronization could start
querying. If they executed a query within this window, virtual docs
would not be visible because the plugin manager would not yet have a
compiler to return to the server. Similarly, if clients queried the
v1/policies API they would see the raw policy contents but no AST
(since the latter is retrieved from the compiler.)

To remove the race condition the plugin manager simply registers the
trigger before starting any of the plugins. This ensures that it sees
all changes made by any of the plugins.

Fixes #2343

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-27 15:26:43 -07:00
Charlie Egan 5549249fa7 Fix minor reference typo
Signed-off-by: Charlie Egan <charlieegan3@users.noreply.github.com>
2020-04-27 12:34:51 -07:00
Arun Kumar 99be0630d3 Fixed home page learn more buttons
Signed-off-by: Arun Kumar <arunbsar@gmail.com>
2020-04-27 12:34:02 -07:00
Torin Sandall 05761a8dec server: Fix data races around state modified on reload
This commit fixes another data race found by running go test
-race. The server was reading state modified by the trigger handler
without using a mutex or being inside of a transaction.

Note, we should think about how we can move away from relying on the
server transactions to implement critical sections in the server as
this is bound to bite us in the future.

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-24 16:22:53 -04:00
Torin Sandall 3124aa7928 topdown: Fix panic in partial eval test of uuid built-in
The recent copy propagation fix requires the compiler during partial
evaluation. In the new uuid built-in function test case, the compiler
was not being provided so it caused a panic.

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-24 09:46:12 -04:00
Patrick East c79dc60d5d topdown: http.send cache test use new test server
The older version of the cache testing for http.send didn't use
separate t.Run()'s for each case so it made it hard to see which case
was broken if the number of requests didn't match up with expected.
It also re-used the same test server which seemed to cause some
timing issues.

Signed-off-by: Patrick East <east.patrick@gmail.com>
2020-04-24 07:14:27 -04:00
Patrick East 1365a38ae5 topdown: Fix for unsafe negated query in partial results
In copy propagation we ran into issues where an expression was negated
and required a variable in it to show up in a non-negated expression
to be safe.

What we do now is expose some of the safety helpers from the ast
package and check if any of the removed expressions provided safety
to any vars left in the result (that are otherwise unsafe). If they
do we will re-add the expression.

This isn't done until after computing the initial result to ensure
that we get the minimal set of expressions. If we attempt to check
eagerly, before we kill/remove any, we might let one remain only to
later add an expression to the result that makes it no longer used.

The current pattern of removing as aggressively as possible and then
taking another pass over the removed expressions to re-add seems to
be the most optimal approach.

Fixes: #2045
Signed-off-by: Patrick East <east.patrick@gmail.com>
2020-04-24 07:14:27 -04:00
Patrick East 43cbc8be71 topdown/copypropagation: Rename "bindings" and use ValueMap
The older "bindings" name was making things more complicated as it
overlapped with the unionfind, which is also variable bindings..

What the "bindings" really were is a set of equality expressions which
had been removed from the query and needed to be accounted for later
on.

Just by renaming this it makes the code way easier to read through
and reason about.

Now that the unionfind can support more than just var's we can extend
the removed expressions to use any value as well. This doesn't
implement any of these changes yet, just setting the state for it in
the future.

Signed-off-by: Patrick East <east.patrick@gmail.com>
2020-04-24 07:14:27 -04:00
Patrick East a36e124842 topdown/copypropagation: unionfind now uses ast.Value
The underlying code will work in the same way, it doesn't affect the
algorithm. We just were limiting to ast.Var before because of the
initial use case only wanting to group variables.

Signed-off-by: Patrick East <east.patrick@gmail.com>
2020-04-24 07:14:27 -04:00
Patrick East 0c9f5aa4af topdown/copypropagation: Add unit tests for unionfind
Signed-off-by: Patrick East <east.patrick@gmail.com>
2020-04-24 07:14:27 -04:00
Patrick East f3d663e85d topdown/copypropagation: Move unionfind to new file
Just moving the code, no other changes.

Signed-off-by: Patrick East <east.patrick@gmail.com>
2020-04-24 07:14:27 -04:00
Torin Sandall 965723c736 topdown: Fix data race in the http.send timeout tests
Also, since these tests are slow, move them into another flag and gate
them with a build flag.

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-23 16:29:50 -04:00
Younes Zahidi 49d35ce75b docs: Add additional kong integration
Kong version 1.0.0 introduced the Plugin Development Kit and updated all bundled plugins to use version 1.0 of the PDK.
All entities and concepts marked as deprecated in Kong 0.x have been removed in Kong 1.0 and a final version (0.15) that support plugins for Kong 0.x series as been released at the same time.

The TravelNest implementation has been done for Kong v0.13.x and has not been updated to work with versions > 1.0.x

This new implementation follows the new plugins project structure and has been tested with Kong 2.0.0.

Signed-off-by: Younes Zahidi <12915382+zahiyo@users.noreply.github.com>
2020-04-23 12:20:03 -07:00
Torin Sandall 88bb42be85 topdown: Add test coverage for new uuid built-in function
Also, prevent uuid function from being partially evaluated for
now. Eventually we will need better controls in place for this kind of
thing but for the time being it is safe to exclude from partial
evaluation like we do with http.send and time.now_ns.

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-22 15:35:41 -07:00
Teemu Koponen 432bcb9c60 wasm: Set implementation built on a chained hash set.
Signed-off-by: Teemu Koponen <koponen@styra.com>
2020-04-22 15:35:19 -07:00
Teemu Koponen c1678c068d wasm: Object implementation built on a chained hash table.
Signed-off-by: Teemu Koponen <koponen@styra.com>
2020-04-22 15:35:19 -07:00
Patrick East 213ec5ad7a build: Remove packaging step from build-release.sh
We will be doing the packaging steps in a separate workflow from the
current one that uses build-release.sh. For now this means just
removing the calls to make the deb file from the script.

Signed-off-by: Patrick East <east.patrick@gmail.com>
2020-04-22 07:11:02 -04:00
Krsna Mahapatra bd3a25ab2d Add deb packaging
Signed-off-by: Krsna Mahapatra <keshto@gmail.com>
2020-04-21 16:25:17 -07:00
Torin Sandall caa4072279 ast: Rename UUID symbol to avoid confusion
Also remove the prefix on the cache key since it's not needed (the
type alias is enough to prevent conflicts, e.g., uuidCachingKey("X")
!= "X".)

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-21 14:25:28 -04:00
Torin Sandall bbcbbe175a topdown: Fix panic in array.slice due to incorrect clamping
The built-in function implemented clamping in an odd way. With this
change, the stopIndex will always be [0, len(arr)] and the startIndex
will always be [0, clamped_stopIndex].

Fixes #2320

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-21 14:11:18 -04:00
Torin Sandall dd04920b73 docs: Update philosophy page to discuss the document model
When we refreshed the docs last year we removed the page titled 'how
OPA works' and in the process lost the description of the OPA document
model that explains the base and virtual document concepts. Since then
a bunch of people have referred back to those docs after getting
started.

This change re-introduces the model and provides an explanation of
_why_ the model exists.

This change also replaces the infographic with text containing the
same content as that feels a bit more inline with the rest of the
page.

Fixes #2284

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-21 08:38:45 -04:00
Rene Klootwijk dbc5c16cd4 A built-in command to create a version 4 uuid
Signed-off-by: Rene Klootwijk <rene.klootwijk@rencon.nl>

A built-in command to create a version 4 uuid

Signed-off-by: Rene Klootwijk <rene.klootwijk@rencon.nl>

A built-in command to create a version 4 uuid

Signed-off-by: Rene Klootwijk <rene.klootwijk@rencon.nl>

Removed external dependency and created helper for uuid

Signed-off-by: Rene Klootwijk <rene.klootwijk@rencon.nl>

Added idempotency for the uuid.rfc4122() command

Signed-off-by: Rene Klootwijk <rene.klootwijk@rencon.nl>
2020-04-20 19:42:03 -07:00
Arun Kumar 6a20655604 Fixed home page hover issue #2206
Signed-off-by: Arun Kumar <arunbsar@gmail.com>
2020-04-17 14:07:37 -07:00
mikaelcabot 95da8ed163 fix(style): opa fmt with else keyword
Fixes: #2299

Signed-off-by: mikaelcabot <mikaelcabot@gmail.com>
2020-04-17 14:06:28 -07:00
Torin Sandall e21a93330e Revert "plugins: Additive updates to services when discovery enabled"
This reverts commit d67c0f7f13.

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-17 13:03:08 -04:00
Torin Sandall 5fd5349f13 docs: Add guidance around resource utilization
There are many ways this question can be answered so this is not a
final answer but instead a starting point. In the future we can expand
on this section to include more use-case specific resource utilization
guidance however for the time being this is a good start.

Fixes #1601

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-17 07:32:04 -04:00
Torin Sandall 23e2a51fbc topdown: Fix trace to set location on notes
Previously the note events would not have a location on them which
mean they were difficult to track down (you would have to grep for the
message and hope it shows up.) With this change we just include the
location on notes like all other events.

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-17 06:51:28 -04:00
Torin Sandall a90187a06b cmd/test: Deprecate -l flag to show line numbers
This flag and the underlying logic to report the line number is not
needed anymore. We have a better solution with the location
info on the pretty tracer.

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-17 06:51:28 -04:00
Torin Sandall d15195df10 Report location info on pretty traces
Recently we improved the trace pretty printing to include location
information on events. Unless there's a good reason we should use this
tracer printing throughout.

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-17 06:51:28 -04:00
Ashutosh Narkar d67c0f7f13 plugins: Additive updates to services when discovery enabled
Earlier with discovery enabled updates to the 'services' configuration
was not allowed to protect against accidental changes to the discovery service
itself. Since adding new services could be useful, this change allows modifications
to the 'services' configuration. The only exception is that the service used to download
the discovery bundle cannot be modified.

Fixes #2058

Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
2020-04-16 20:46:14 -07:00
Patrick East 71454de28a Update dev docs
Adds some additional links on some workflow steps and removes an older
unused section.

There is also a little bit more detail on the desired commit message
format.

Signed-off-by: Patrick East <east.patrick@gmail.com>
2020-04-16 16:10:53 -07:00
marco 5e6a138559 docs: Fix the EBNF for the '=' terminal symbol
The rule-body non-terminal symbol did not quote the '=' symbol.

Signed-off-by: marco <marco@styra.com>
2020-04-16 13:21:19 -07:00
Torin Sandall 6ff4af4e80 docs: Fix example prometheus config
The 'global_scrape' key was renamed to 'global' at some point.

Fixes #2277

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-16 12:46:57 -04:00
Torin Sandall 6a222dcec9 Update governance model to reflect areas of expertise
The previous governance model did not account for areas of expertise
like gatekeeper or other sub-projects. As the OPA project grows, it is
necessary to clarify responsibilities and ownership of different parts
of the project.

This commit updates the governance model to reflect (1) that we do not
hold votes on individual PRs (this was cargo-culted from an existing
CNCF document at the time and is not relevant) (2) organization voting
will be used to resolve conflicts and (3) maintainers statuses and
areas of expertise.

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-16 08:09:26 -04:00
Torin Sandall 525afe9d52 metrics: Include Add(uint64) on Counter object
This change is not backwards compatible but it is only a minor
inconvenience for anyone who has re-implemented metrics.Metrics (which
feels highly unlikely anyway.)

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-14 09:50:28 -04:00
Torin Sandall 2a8e2936f7 ast: Refactor output var analysis
The ouput vars analysis had duplicated code paths to deal with old
built-in calling conventions where any arg could be an output. Since
we don't support this anymore at the built-in declaration level, we
can remove the special cased logic for it.

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-14 09:26:22 -04:00
Torin Sandall fbb0a39f22 ast: Fix bug in output var analysis and add unit tests
There was a bug in the output var analysis whereby the implementation
would visit references inside closures when it should have
stopped. This was not seen because the output var analysis was
primarily used by the body safety check that special cases closures
and avoids thep problem. As part of the fix we now have a set of unit
tests for the functionality.

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
2020-04-14 09:26:22 -04:00
Patrick East 25029dffe9 Prepare v0.20.0 development
Signed-off-by: Patrick East <east.patrick@gmail.com>
2020-04-13 17:26:31 -07:00