From ea5cdbf6f1094eccb9217e1cc91c642b13589936 Mon Sep 17 00:00:00 2001 From: Tristan Swadell Date: Fri, 22 Dec 2017 14:17:13 -0800 Subject: [PATCH] Rego v2 proposal updates --- design/REGO_V2_PROPOSAL.md | 377 +++++++++++++++++++++++++------------ 1 file changed, 253 insertions(+), 124 deletions(-) diff --git a/design/REGO_V2_PROPOSAL.md b/design/REGO_V2_PROPOSAL.md index 8aed28c45c..cc1169f97e 100644 --- a/design/REGO_V2_PROPOSAL.md +++ b/design/REGO_V2_PROPOSAL.md @@ -1,7 +1,7 @@ Rego v2 - Proposal Authors: Tristan Swadell, Tim Hinrichs, Torin Sandall -Last-Modified: 2017-10-20 +Last-Modified: 2017-12-15 # Goals @@ -17,20 +17,19 @@ The user-experience for policy enforcement depends heavily on the policy language and what concepts the user must understand to use that language. The proposed concepts thus far are: -* **Policy** - * Functions that provide a typed decision, e.g. boolean, list, map, etc. - * Determine which sub-policies are relevant and their computational - complexity. +* **Rule** + * Functions or constants that provide a typed decision, e.g. boolean, + list, map, etc. + * Determine which sub-rules are relevant and their computational complexity. * May be packaged into modules to support logic reuse and the construction of libraries for specific domains (K8s, service meshes, cloud APIs). * **Trigger** - * Functions that produce side-effects which run after policy. - * May inspect policy context as well as policy invocations/decisions. + * Functions that produce side-effects which run after rules. + * May inspect rule context. * Side-effects may be one of: - * Obligation - synchronously and successfully executes before the - request. - * Promise - asynchronously guaranteed to run if the request succeeds. + * Obligation - guaranteed to execute if the request succeeds, may be + synchronous (precondition) or asynchronous (promise).` * Advice - best-effect asynchronous execution on request success. * **Context** @@ -39,82 +38,103 @@ proposed concepts thus far are: The proposed language has the following properties: -* Policies are packaged into modules -* At most one default package. -* At most **one `main()` `@policy`** per module. +* Rules are packaged into modules +* The absence of qualified package name indicates the default package. +* Rules are evaluated by name or using the `main()` entry point if no name is + specified. * Conditions determine applicable decisions. * Conditions may be **hierarchical**. -* Conditions may invoke other policies. +* Conditions may invoke other rules. * Decisions are **evaluated in order, first decision wins**. -* A policy must always make a decision. +* A rule must always make a decision. * Conditions, decisions, local declarations, and triggers may invoke functions. * Functions are **side-effect free**. -* Triggers run after policies and may be cross-cutting. +* Triggers run after rules and may be cross-cutting. -# Policies +# Rules -The policy declaration provides a named entry point for evaluation and -composition. The signature of the policy declares the context to be provided +The rule declaration provides a named entry point for evaluation and +composition. The signature of the rule declares the context to be provided upon evaluation, and the body is a collection of conditions and decisions. -The general form of policy is effectively a decorated function with the -relevant portion of the grammar defined in the Language section presented here. -Please note that the definition of 'expr' is taken from the -[Common Expression Language](http://github.com/google/cel-spec) (CEL): +The general form of rule is effectively a decorated function or constant with +the relevant portion of the grammar defined in the Language section presented +below. Please note that the definition of `expr` and `literal` are taken from +the [Common Expression Language](http://github.com/google/cel-spec) (CEL): ``` function_decl - := decorator? 'function' id '(' arg_list? ')' ('{' statement+ '}')?; + := decorator? 'function' id '(' arg_list? ')' ('{' statement+ '}')? + ; decorator - := '@' id; + := '@' id decorator_arg_list? + ; +decorator_arg_list + := '(' id '=' literal (',' id '=' literal)* ')' + ; arg_list - := id (',' id)*; + := id (',' id)* + ; statement - := condition_expr | const_decl | return_expr | comprehension_expr; + := condition_expr + | const_decl + | return_expr + | comprehension_expr + | block_expr + ; condition_expr - := filter_expr '{' statement+ '}'; + := filter_expr '{' statement+ '}' + ; filter_expr - := 'if' expr; + := 'if' expr + ; const_decl - := id assign_expr ';'; + := decorator? id assign_expr? ';' + ; assign_expr - := '=' (expr | comprehension_expr); + := '=' (expr | comprehension_expr) + ; return_expr - : 'return' (expr | comprehension_expr) ';'; + : 'return' (expr | comprehension_expr) ';' + ; comprehension_expr := iter_expr | '{' (expr '|')? iter_expr '}' - | '[' (expr '|')? iter_expr ']'; + | '[' (expr '|')? iter_expr ']' + ; iter_expr := 'for' id (',' id)? 'in' expr filter_expr? statement?; - | 'for' id (',' id)? 'in' expr filter_expr? ('{' statement+ '}')?; + | 'for' id (',' id)? 'in' expr filter_expr? ('{' statement+ '}')? + ; +block_expr + := decorator? '{' statement+ '}'; ``` -Policies are functions that return a decision and are marked with the `@policy` -decorator. The signature of the policy indicates the context required from the -caller. The body of the policy may contain any number of conditions, +Rules represents a decision and are marked with the `@rule` decorator. A rule +may either be a constant or a function. A rule may be a constant when the +decision may be derived entirely from module or system provided context. For +function rules, the signature of the function indicates the context required +from the caller. The body of the rule may contain any number of conditions, decisions, and local declarations. The decorator indicates that the function -is intended to be used for policy decisions as opposed to simply being a -function used for resolving intermediate answers. Likewise, the decorator -ensure the policy symbol and its decision are tracked and available for use -within `@trigger` functions. +or value is intended to be used for decisions rather than simply being an +intermediate piece of logic. ``` -@policy function userSalary(resource, user) { +@rule function userSalary(resource, user) {   match_result = resource.match('users/{target_user}/salary'); return user == match_result.group.target_user; } ``` -Policies may be imported and leveraged within policy decisions. Note the -inclusion of the policy within a package and how this affects function -identifier resolution. +Rules may be imported and leveraged within rule decisions. Note the inclusion +of the rule within a package and how this affects function identifier +resolution. ``` package acme; import acme.hr; -@policy function readUserSalary(request, resource, user) { +@rule function readUserSalary(request, resource, user) {   match_result = resource.match('users/{target_user}/salary');   if (match_result.matches() && request.method == 'get') {       target_user = match_result.groups.target_user; @@ -126,13 +146,13 @@ import acme.hr; } ``` -Policies may have multiple return statements in order to enforce allow / deny -semantics (in the case of binary policies), or simply different variations on -an affirmative policy decision, such as whether to return a list of honey-pot +Rules may have multiple return statements in order to enforce allow / deny +semantics (in the case of binary rules), or simply different variations on +an affirmative rule decision, such as whether to return a list of honey-pot servers versus a valid list of servers. ``` -@policy function readUserSalary(request, resource, user) { +@rule function readUserSalary(request, resource, user) {   match_result = resource.match('users/{target_user}/salary');    if (match_result.matches() && request.method == 'get') {       target_user = match_result.groups.target_user; @@ -148,17 +168,17 @@ servers versus a valid list of servers. } ``` -Policies may be composed. The example below indicates how reading and listing a -salary are lumped into a single policy decision. +Rules may be composed. The example below indicates how reading and listing a +salary are lumped into a single decision. ``` -// Policies may be composed. -@policy function viewSalary(request, resource, user) { +// Rules may be composed. +@rule function viewSalary(request, resource, user) {   return queryDepartmentSalaries(request, resource, user) || readUserSalary(request, resource, user); } -@policy function queryDepartmentSalaries(request, resource,user) { +@rule function queryDepartmentSalaries(request, resource, user) {   match_result = resource.match('departments/{department}/salaries')   if match_result.matches() && request.method == 'list' {     department = match_result.groups.department; @@ -171,12 +191,29 @@ salary are lumped into a single policy decision. // ... readUserSalary ... ``` +Rules may also be written as assignments or as anonymous blocks: + +``` +// Assignment style rule which supports named evaluation. +@rule authenticated = request.auth != null ? allow : defer; + +// Block style rules are useful when all @rule statements will be evaluated +// together. The last statement in the block is treated as the rule decision. +@rule { + // Note: the authenticated rule included by name. + // This rule allows requests for resources marked public, otherwise denies. + return !authenticated && resource.name.contains('/public/') + ? allow : deny; +} +``` + ## Conditions Conditions are [Common Expression Language](https://github.com/google/cel-spec) (CEL) expressions and evaluate to a boolean outcome. A condition may be used to select applicable rules or to make an effect or trigger conditional. -Conditions may also be used to filter list and map entries within a for-in expression. +Conditions may also be used to filter list and map entries within a `for-in` +expression. Declarations within a condition are within its block scope and may be shadowed by declarations in nested conditions. Once a declaration has been assigned, it @@ -184,36 +221,53 @@ cannot be reassigned. ## Decisions -A decision is simply a return statement within a policy function. The types of -all decisions must agree. For authorization policies, the decisions will -typically be boolean with a default decision of return false to indicate deny -by default semantics. +A decision is a return statement within an `@rule`. The decision types across +`@rule` declarations *should* agree. In the case of multiple `@rule`s being +evaluated where the outputs of each do not agree, the decision is considered +dynamically typed. For authorization rules, decisions are often boolean or +three-valued. In the three-value case the decision may be one of: `allow`, +`deny`, and `defer` where the last decision type defers the `allow` or `deny` +decision to subsequent `@rule`s. + +The decision semantics when multiple `@rule`s are evaluated depends on the +conflict resolution algorithm used when executing rules. + +Note: decision resolution across `@rule`s are as yet undefined, but will be +addressed in a future update to the proposal. # Triggers -Triggers are functions that emit obligations, promises, and advice to be -performed based on the policy decision. The functions emit contextual -information about the operation to be performed and the impact on the -overall request behavior. An obligation must be performed synchronously and -succeed prior to servicing the request. +Triggers are functions that emit obligations and advice to be performed based +on the decision. The functions emit contextual information about the operation +to be performed and the impact on the overall request behavior. The terms +obligations and advice align with the concepts introduced within +[XACML](xacml.org). Although XACML does not specify the order of execution or +conflict resolution behavior among obligations, this can radically affect +request behavior. Future revisions of this spec should address the different +kinds of obligations that may be emitted and conflict resolution strategies to +be employed during execution, but for now they are broken down into two +categories: synchronous and asynchronous. Synchronous obligations may include +preconditions, whereas async obligations would be considered promises. Advice, +on the other hand, should always be considered asynchronous and best-effort. -In the policy examples, the decision is binary, so the first argument to the +In the rule examples, the decision is binary, so the first argument to the trigger has been labelled allow although it could be any type of decision. The second argument is a dynamic object representing context supplied to the -policies responsible for the decision. +rules responsible for the decision. ``` -// Triggers are evaluated after policy decisions and have a void return. -@trigger function onSalaryRequest(allow, ctx) { - // Triggers may inspect the policy signatures invoked and their outcomes. - if (ctx.request.method in ['get', 'list'] && !viewSalary) { -   logger.log(ctx.user + " denied view salary request" + ctx.resource); +// Triggers are evaluated after rule decisions and have a void return. +@trigger function onSalaryRequest(decision, ctx) { + // Triggers may invoke rules without affecting the rule outcome. + if (ctx.request.method == 'list' + && !queryDepartmentSalaries(ctx.request, ctx.resource, ctx.user)) { +   logger.log(ctx.user + " denied view salary request " + ctx.resource);   } // Alternatively, triggers may simply inspect context to make a decision.   match_result = ctx.resource.match('users/{target_user}/**'); target_user = match_result.groups.target_user; -  if (ctx.user != target_user && allow) { +  if (ctx.user != target_user && decision == allow) {   logger.log(ctx.user + " perform a salary action on " + target_user);   } } @@ -221,7 +275,36 @@ policies responsible for the decision. Triggers are the only functions that can and must have a void return type. They must not be referenced within non-trigger functions, though they may invoke other -non-policy functions including other triggers. +non-rule functions including other triggers. + +# Tests + +Being able to verify the correctness policy-related logic is of paramount +importance. As is the ability to pose ad hoc queries with partial state. To this +end we include `@test` as supported decorator and introduce the `with-as` clause +to assist with partial state bindings required for both adhoc queries and for +function mocking. + +Note: the following is under review and not yet reflected in the grammar. + +``` +@rule function user_owned_action(auth, resource) { + result = resource.matches('documents/{owner}/**') + return (result.owner == auth.uid + || resource.owner in user_groups(auth.principal)); +} + +@extern function user_groups(user); + +function mock_user_groups(user) { … } + +@test function group_check() { + with user_groups as mock_get_group_users { + assertTrue(user_owned_action({principal: 'me'}, 'documents/my-group')) + assertFalse(user_owned_action({principal: 'me'}, 'documents/their-group')) + } +} +``` # Context @@ -243,8 +326,8 @@ syntax = 'rego.v2'; ctx = {resource: resource().name, resource_owner: resource().owner}; -// Allow user-owned reads, with user and request provided as a policy argument. -@policy function main(user, request) { +// Allow user-owned reads, with user and request provided as a rule argument. +@rule function allow_user_reads(user, request) { if (request.method in ['get', 'list']) { return (request.auth.claims.email == user || ctx.resource_owner == user @@ -273,8 +356,8 @@ syntax = 'rego.v2'; ctx = {'resource': db.resource().name, 'resource_owner': db.resource().owner}; -// Allow user-owned reads, with user and request provided as a policy argument. -@policy function main(user, request) { +// Allow user-owned reads, with user and request provided as a rule argument. +@rule function allow_user_reads(user, request) { if (request.method in ['get', 'list']) { return (request.auth.claims.email == user || ctx.resource_owner == user @@ -284,20 +367,22 @@ ctx = {'resource': db.resource().name, } ``` -When no additional context is necessary beyond what is available within the -module, the `main()` declaration may be omitted with the contents of the module -simply being the function body. +The example below shows how `acme.db` provides a library of context and +functions for use with @rule statements. The `@extern` decorator is equivalent +to a forward declaration, both to serve as documentation for what exists, but +also to be consumed during type-checking to ensure the system context and +function hooks are being used correctly within @rule statements. ``` package acme.db; syntax = 'rego.v2'; +@extern request; @extern function resource(); -@extern function request(); @extern function query(document_name); input = { - 'method': request().method, - 'user': request().auth.claims.email, + 'method': request.method, + 'user': request.auth.claims.email, 'resource': resource() } @@ -318,7 +403,7 @@ syntax = 'rego.v2'; // Allow user-owned reads, with context information provided by functions and // constants provided by acme.db in the form of module or extern declarations. -@policy { +@rule { target_user = db.resourceMatch('users/{target_user}/salary').target_user; return db.permission.read && (db.input.user == target_user @@ -356,13 +441,20 @@ function_decl := decorator? 'function' id '(' arg_list? ')' ('{' statement+ '}')? ; decorator - := '@' id + := '@' id decorator_arg_list? + ; +decorator_arg_list + := '(' id '=' literal (',' id '=' literal)* ')' ; arg_list := id (',' id)* ; statement - := condition_expr | const_decl | return_expr + := condition_expr + | const_decl + | return_expr + | comprehension_expr + | block_expr ; condition_expr := filter_expr '{' statement+ '}' @@ -371,17 +463,13 @@ filter_expr := 'if' expr ; const_decl - := id assign_expr ';' + := decorator? id assign_expr? ';' ; assign_expr - := '=' expr + := '=' (expr | comprehension_expr) ; return_expr - := 'return' expr ';' - ; -expr - := conditional_expr - | comprehension_expr + : 'return' (expr | comprehension_expr) ';' ; comprehension_expr := iter_expr @@ -389,8 +477,11 @@ comprehension_expr | '[' (expr '|')? iter_expr ']' ; iter_expr - := 'for' id (',' id)? 'in' expr filter_expr? statement?; - | 'for' id (',' id)? 'in' expr filter_expr? ('{' statement+ '}')? + := 'for' id (',' id)? 'in' expr filter_expr? statement? + | 'for' id (',' id)? 'in' expr filter_expr? ('{' statement+ '}')? + ; +block_expr + := decorator? '{' statement+ '}' ; conditional_expr := or_expr ('?' or_expr ':' expr)? @@ -403,12 +494,12 @@ and_expr ; relation_expr := calc_expr - | relation_expr ('<'|'<='|'>='|'>'|'=='|'!='|'in') relation_expr + | relation_expr ('<'|'<='|'>='|'>'|'=='|'!='|'in') relation_expr ; calc_expr := unary_expr - | calc_expr ('*'|'/'|'%') calc_expr - | calc_expr ('+'|'-') calc_expr + | calc_expr ('*'|'/'|'%') calc_expr + | calc_expr ('+'|'-') calc_expr ; unary_expr := compound_expr @@ -417,18 +508,18 @@ unary_expr ; compound_expr := primary_expr - | compound_expr '.' id - | compound_expr '.' id '(' expr_list? ')' - | compound_expr '[' expr ']' - | qualified_id '{' field_inits? '}' + | compound_expr '.' id + | compound_expr '.' id '(' expr_list? ')' + | compound_expr '[' expr ']' + | qualified_id '{' field_inits? '}' ; primary_expr := '.'? id | '.'? id '(' expr_list? ')' - | '(' expr ')' - | '[' expr_list? ']' - | '{' map_inits? '}' - | literal + | '(' expr ')' + | '[' expr_list? ']' + | '{' map_inits? '}' + | literal ; expr_list := expr (',' expr)* @@ -451,7 +542,7 @@ this may be introduced in the future. ## Constants Constants are identifiers associated with an expression. They are useful for -clarifying the logical relationship between components of a policy decision. +clarifying the logical relationship between components of a rule decision. Constants referenced within expression will evaluate in the same manner as though they were written inline into the statement. @@ -459,11 +550,11 @@ though they were written inline into the statement. Functions are simply collections of logical statements. Functions decorated with `@trigger` must have a void return. All other functions, whether decorated -as a `@policy` or not, must have a return value. A function body consists of +as a `@rule` or not, must have a return value. A function body consists of any number of assignments and a single return expression. Argument and return types are inferred based on usage. -All Functions are idempotent. Given the same input, the functions must return +Functions are idempotent. Given the same input, the functions must return the same output (or in the case of triggers emit the same output). The is true for both local and extern functions. At present functions do not support overloads, though this may change in future iterations of this proposal. @@ -471,14 +562,20 @@ overloads, though this may change in future iterations of this proposal. The general form of a function is as follows: ``` -function_decl = decorator? 'function' id '(' arg_list? ')' ('{' statement+ '}')? -decorator = '@' id; -arg_list = expr (',' expr)* +function_decl + := decorator? 'function' id '(' arg_list? ')' ('{' statement+ '}')? + ; +decorator + := '@' id decorator_arg_list? + ; +decorator_arg_list + := '(' id '=' literal (',' id '=' literal)* ')' + ; ``` -The set of supported decorators is limited to `@extern`, `@policy`, and -`@trigger`. When a function may is annotated with @extern, it must be supplied -at runtime as part of the policy evaluation context. For example: +The set of supported decorators is limited to `@extern`, `@rule`, `@trigger`, +and `@test`. When a function may is annotated with @extern, it must be supplied +at runtime as part of the rule evaluation context. For example: ``` package db; @@ -497,10 +594,23 @@ local functions rather inline within a rule, but this is not a requirement. The general form of a comprehension is as follows: ``` -list_comprehension = '[' expr | for_in_expr ']' -set_comprehension = '{' expr | for_in_expr '}' -for_in_expr = 'for' id(',' id)? 'in' expr filter? ('{'? for_in_expr '}'?)? -filter = 'if' expr +comprehension_expr + := iter_expr + | '{' (expr '|')? iter_expr '}' // set, map comprehension + | '[' (expr '|')? iter_expr ']' // list comprehension + ; +iter_expr + := 'for' id (',' id)? 'in' expr iter_op_expr? statement? + | 'for' id (',' id)? 'in' expr iter_op_expr? ('{' statement+ '}')? + ; +iter_op_expr + := filter_expr + | 'all' expr + | 'any' expr + ; +filter_expr + := 'if' expr + ; ``` A set comprehension differs from a list comprehension only in the sense that @@ -540,12 +650,30 @@ result. The result type of the for-in is determined by the surrounding braces. ``` // exists one [ for e in list if e < 10 ].size() == 1 -// exists -[ for e in list if e.matches('hello world') ].size() > 1 -// all -{ for k, v in map if k.startsWith('shared') && v > 1 }.size() == map.size() + +// exists at least two unique values, note the `{}` surrounding the `for-in` +// indicates this is the construction of a set from a list. +{ for e in list if e.startsWith('t') }.size() > 2 ``` +There are also two special operators which can be used to make qualitative +inferences about the elements within an aggregate type, `all` and `any`. The +primary difference between these operators and the filter expressions (or +even standard comprehensions) is that they are accumulator functions with +the same semantics as the logical `&&` and `||`, meaning these operators +can absorb errors. + +``` +// Any element exists in this list +for e in list any e != 'bad_candidate' + +// All elements in this list must be good candidates +for e in list all e == 'good_candidate' +``` + +The `any` and `all` yield boolean outcomes and are thus specialized reducing +functions capable of absorbing errors in the same manner as hand-rolled code. + ### Map It is very common for developers to massage data from a variety of sources in @@ -599,3 +727,4 @@ Importing a module makes all symbols within that module accessible by either the fully qualified module name, or the simple module name (the last fragment of a qualified package identifier). +