Parsing is generally fast, so this mainly improves performance
of creating big bundles with many Rego files in them. For Regal's
embedded bundle, loading it from memory would previously take 16
ms on my laptop, and now it takes 9 ms. There are other things
in this process that could be concurrent too, like JSON unmarshalling
of multiple data files. But starting with parsing modules.
This PR adds `errgroup` as a direct dependency (previously indirect)
as it is a nicer way to work with wait groups, and one that can be
useful elsewhere in the codebase (like in the compiler).
Also, and as usual, went off on a bit of a tangent refactoring code
related to the bundle build process, and made sure to use some common
helpers in code where available.
Signed-off-by: Anders Eknert <anders.eknert@apple.com>
Previously we had it in an internal package but used by a public API,
which basically means it can't actually be used outside of OPA.
Initial thinking was that this was an OK situation, but by popular
demand we are making it available to everyone so OPA as a lib users
can use the bundle loading API's.
Fixes: #1840
Signed-off-by: Patrick East <east.patrick@gmail.com>
When loading files we would build the list with, on windows, potential
paths with backslashes rather than forward slashes. This is
problematic because the check to trim the prefix for returning file
descriptors based on the root of the bundle.. On windows it could, in
some cases, fail to do so. This results in additional prefixes on the
loaded data.
Signed-off-by: Patrick East <east.patrick@gmail.com>
Users should be able to pass file:// URLs to any of the
sub-commands. In 3be55ed6 the eval and fmt sub-commands were not
updated to accept file:// URLs for the input file and normal paths
(respectively).
This commit just moves the unexported cleanFileURL function from the
loader package into it's own internal package so that it can be shared
in OPA.
Signed-off-by: Torin Sandall <torinsandall@gmail.com>
This introduces a new interface for loading directories. It is
only used for bundles right now, but it is not specific to them.
A version for tarballs and just normal filesystem directories is
added, and the bundle loader code is updated to use the right one.
Signed-off-by: Patrick East <east.patrick@gmail.com>
Anders Eknert