mirror of
https://github.com/open-policy-agent/opa.git
synced 2026-08-20 15:31:14 -06:00
Anders Eknert
7fa6165c27
We do a lot of `walk`-ing in [Regal](https://docs.styra.com/regal). So much that it's by far the single most expensive operation. That means any optimization of the `walk` built-in function will be a win for us. Seeing as we rarely make use of the `path` component when `walk`-ing through AST inputs, I was curious to see if there was any optimization we could take when the path is a wildcard assignment, and as such clearly marked as unimportant. Turns out there is. Since the return value is provided in the operators list, we can check the value provided for the `path` part of the assigned array, and if it's a wildcard (`_`) skip path construction entirely. Example: ```rego walk(input, [_, value]) ``` This greatly simplifies the walk, and the performance gains are substantial. Traversing a ~7MB AST: **main** ```shell $ opa bench -d p.rego -i objects.json data.p.w +-------------------------------------------+---------------+ | samples | 6 | | ns/op | 168806625 | | B/op | 197364318 | | allocs/op | 3855327 | | histogram_timer_rego_query_eval_ns_75% | 169968114 | | histogram_timer_rego_query_eval_ns_90% | 170513459 | | histogram_timer_rego_query_eval_ns_95% | 170513459 | | histogram_timer_rego_query_eval_ns_99% | 170513459 | | histogram_timer_rego_query_eval_ns_99.9% | 170513459 | | histogram_timer_rego_query_eval_ns_99.99% | 170513459 | | histogram_timer_rego_query_eval_ns_count | 6.00 | | histogram_timer_rego_query_eval_ns_max | 170513459 | | histogram_timer_rego_query_eval_ns_mean | 168789611 | | histogram_timer_rego_query_eval_ns_median | 168924020 | | histogram_timer_rego_query_eval_ns_min | 166685000 | | histogram_timer_rego_query_eval_ns_stddev | 1239390 | +-------------------------------------------+---------------+ ``` **no-path-walk** ```shell $ opa bench -d p.rego -i objects.json data.p.w +-------------------------------------------+--------------+ | samples | 21 | | ns/op | 50629984 | | B/op | 38018790 | | allocs/op | 1025211 | | histogram_timer_rego_query_eval_ns_75% | 51239562 | | histogram_timer_rego_query_eval_ns_90% | 51540933 | | histogram_timer_rego_query_eval_ns_95% | 51674420 | | histogram_timer_rego_query_eval_ns_99% | 51688208 | | histogram_timer_rego_query_eval_ns_99.9% | 51688208 | | histogram_timer_rego_query_eval_ns_99.99% | 51688208 | | histogram_timer_rego_query_eval_ns_count | 21.0 | | histogram_timer_rego_query_eval_ns_max | 51688208 | | histogram_timer_rego_query_eval_ns_mean | 50611103 | | histogram_timer_rego_query_eval_ns_median | 50871459 | | histogram_timer_rego_query_eval_ns_min | 49518833 | | histogram_timer_rego_query_eval_ns_stddev | 748688 | +-------------------------------------------+--------------+ ``` The real-world impact is not as dramatic, since we aren't *just* walking, but normally need to actually **do** something with the values returned, but consistently shaving off about 13% eval time when linting one of the largest policy libraries isn't bad at all: **Regal main** ```shell go run main.go lint ~/tmp/kics/assets 162.16s user 6.04s system 593% cpu 28.362 total ``` **Regal walk-no-path** ```shell go run main.go lint ~/tmp/kics/assets 145.51s user 5.01s system 597% cpu 25.176 total ``` Signed-off-by: Anders Eknert <anders@eknert.com>