Previously, nested references were not allowed. This was particularly annoying
when rules were used to define constant values because it required an
intermediate variable to store the constant value in the current scope (which
could then be used in the reference).
Now, nested references are allowed and the compiler and evaluation engine have
been updated to support them. Specifically, the evaluation engine will
recursively evaluate nested references before the outer most reference is
evaluated. The evaluation adds a binding for the nested reference to the
context so that when the containing term or expression is plugged, the nested
references are replaced with the referred value.
The compiler has been updated to include body safety, reordering, and
recursiong tests involving nested references.
The unset command can be used to undefine rules identified by a variable in
the currently active module. This allows users to workaround the additive
nature of rule definitions.
When comprehensions were added, a post processing step was introduced to
mangle wildcards. There were two separate code paths for loading modules and
one of them was missed. Now there is only one!
Previously, the REPL treated all bodies as queries: the query was evaluated
and for each instance where the query was true, the variable bindings were
displayed. If there were no variables in the body, "true" was printed. If
there were no instances where the query was true, "false" was printed.
Now, the REPL treats single term expressions as a special case under the
assumption that the user wants to see the value of the term. In cases where
the term can only evaluate to a single value (e.g., it's ground), the value of
the term is serialized to JSON and printed. This is useful for initially
learning the language and eventually experimenting with references, e.g.:
> data.servers[0].name
"web"
>
In cases where the term can evaluate to multiple values (i.e., it's
non-ground), the REPL displays one row of output for each instance where the
query is true. Each row contains the bindings of non-wildcard variables as
well as a column for the value of the term itself. This is useful for quickly
evaluating references without defining an intermediate variable, e.g.:
> data.servers[x].name
0,"web"
1,"app"
...
>
Previously, callers were not able to query virtual docs defined by rules that
were written in terms of query inputs that in turn were defined by
references. E.g., a rule dependant on "import request" would be fine, however,
a rule dependant on "import com.example.request" would not.
Callers can now provide input values for imports defined by references. The
REST API constructs the appropriate binding for the variable in the head of
the reference. The binding value is the merged version of all the inputs that
share the same root. Conflicting values are rejected with an error (400)
response.
These changes allow policy authors to namespace their inputs in the same way
they can namespace their base and virtual docs.
Multiple rules may share the same name while defining complete docs as
long as the docs generated by the rules at query time do not conflict. A
conflict is simply multiple values for the same document. The same applies for
object document keys.
This change set does away with the old way of determining which variables are
outputs. Equality is now handled with special care. Outputs that would make a
variable safe by depending on another unsafe variable are no longer included.
As a result, the occurs check in the topdown implementation is no longer
needed. This change was introduced to handle odd cases involving
comprehensions, e.g., x = y, x = [ y | y = 1 ]. In this case, without exluding
unsafe vars, the query would evaluate with x/[1]. This would violate the
semantics, because in the comprehension y/1.
Also, fix bug in reordering whereby potentially unsafe expressions were added
to the reordered body multiple times. This occurred because the expression
would be added once when the preceeding expression made it safe and then again
once the outer loop got it. With the fix, we reprocess the body each time an
expression is added to the reordered body.
Also, move wildcard mangling into post processing step in parser extensions.
Wildcards needs to be mangled after Parse() because otherwise the generated
variable names will reset when handling closures.
Callers need to be able to supply values for global values that virtual docs
depend on. These changes allow callers to supply globals via query param:
GET /v1/data/path/to/some/vdoc?global=<key>:<value>
Where <key> is the import path and <value> is a URL encoded JSON value.
For now the topdown implementation only supports ast.Var values for the key
but in future, references shall also be supported.
A few of the links in the language reference were broken when switching over
to Jekyll. Also, the table formatting was broken by the switch to redcarpet.
Also, update Jekyll configuration to use Redcarpet as Kramdown was inserting
extra spaces into code blocks. This was making the EOF line not get recognized
in the heredocs.
Also, fix missing commas in one of the arch.md policy snippets.
The path handling was broken for patch objects with path = "-".
Refactored Data API test cases as well. They're more concise now and can be
extended easily.
Very little changed, mostly just fixing the formatting of the REPL output.
Eventually we should have automation that runs these examples through the
REPL.
Torin Sandall