It's often useful to be able to create a hierarchical structure in one
shot in storage. Previously this functionality was implemented in the
server, but it's better off implemented in the storage package.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
These changes update the storage layer to support
multi-reader/single-writer txns:
- Writers can read their own writes
- Writers can rollback changes
- Readers only see writes after a successful commit
- Readers can progress during a write txn and only block during a commit
These changes also refactor the trigger interface to better support
transactions.
This is a large change set that contains a few backwards incompatible
changes. Summary of breaking changes:
- Remove storage.Storage in favour of storage.Store interface.
- Remove mount support.
- Remove storage of compiled policies.
- Modify storage.Store to support rollback.
- Modify storage.Store to support raw policy storage.
- Modify storage.Store to support indexing.
- Refactor error codes to use strings instead of ints.
- Simplify error messages throughout.
- Ensure location set on all expressions. There were a couple locations
in the parser/compiler where locations were not being set.
- Fallback to rule location in topdown in case location not set. This
ensures that users get useful locations for API requests with paths
that refer to virtual docs exactly.
Also add Find function to ast.Value. Useful for extracting values
dynamically. Eventually can support JSON pointers.
Fixes#237
These changes refactor the storage layer to use storage.Path instead of ast.Ref
for Read/Write/Begin/Unmount/Mount operations.
Previously, the storage layer used ast.Ref values to refer to locations in
storage. Use of ast.Ref introduced unnecessary complexity for storage plugins
as they had to be aware of various details (e.g., array indices specified as
ast.Number/float64 values, potentially nested references, etc.) that were
unnecessary given that ast.Ref values passed to the storage layer were
intended represent JSON pointers.
These changes also remove the need for storage plugins to be aware of where
they are mounted. That is, the paths passed to the read call will be relative
to the mount point.
The remaining dependencies on the ast package from the storage package are for
(1) indexing and (2) policy storage. It may be possible to further decouple
these packages by revisiting how indexing is done and treating policies as
blobs.
Fixes#159
This is small change that allows callers to issue GET /data and receive the
full extent of OPA's data. Secondly, this allows callers to set the data
in OPA with one request (e.g., for initialization).
Torin Sandall