13 Commits

Author SHA1 Message Date
Johan Fylling a179a24c48 v1 API
All packages, except for `cmd` and `internal`, have been moved into a new `v1` root package.

Old packages are kept for backwards-compatibility reasons. All contained code is replaced with simple type aliases and proxy functions to `v1` implementations.

Old packages default to the Rego v0 syntax, new `v1` packages default to the Rego v1 syntax.

Signed-off-by: Johan Fylling <johan.dev@fylling.se>
2024-12-12 15:27:34 +01:00
Johan Fylling 7bb6dbe36b Preparing for v1 API
Moving (most) source to v1 root package to prepare for v0/v1 API separation.

Signed-off-by: Johan Fylling <johan.dev@fylling.se>
2024-12-12 15:09:03 +01:00
Evan Anderson 84b554c304 Fix flakes in ./topdown/cache
Per discussion in https://github.com/open-policy-agent/opa/pull/7188, do *not* adjust the timing of cache scans to be more frequent; we want to wait at least staleEntryEvictionTimePeriodSeconds between cache scans, even if we hold the lock for a substantial period of time.

Signed-off-by: Evan Anderson <evan@stacklok.com>
2024-12-04 15:44:46 -08:00
Ashutosh Narkar 2c56293695 Add a new inter-query value cache to cache data across queries
This commit adds a new inter-query value cache that built-in
functions can use to cache information across queries.
For example, the `regex` and `glob` builtins can use this
to cache compiled regex and glob match patterns respectively.

The number of entries in the cache can be configured via the OPA
config. By default there is no limit.

Fixes: #6908

Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
2024-09-23 13:03:03 -07:00
Rudrakh Panigrahi f063c90275 Clean expired cache entries periodically
Regularly clean up of cache entries that have expired for a more efficient use of memory.
Introduce two new parameters to tune clean up frequency and threshold for forced FIFO eviction.

Fixes #5320

Signed-off-by: Rudrakh Panigrahi <rudrakh97@gmail.com>
2024-01-09 13:13:30 -08:00
Ashutosh Narkar 2d6ad9da0d topdown: Update cache interface with Clone method
Concurrent evaluation of the http.send builtin for the
same object can sometimes result in the HTTP headers
map being concurrently accessed. This can happen for
example when a key already present in the inter-query
cache needs to be revalidated and multiple routines
may access the HTTP headers at the same time resulting
in a race.

This change adds a new Clone method to cache interface.
The idea is to give each routine its own copy of the cached object
which would mean it has a copy of the headers map and
thus should be able to avoid any sync issues.

Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
2023-06-15 10:23:27 -07:00
Aleksander 600899aaa2 Fix memory leak in interquery cache
Signed-off-by: Aleksander <Alekken@live.no>
2022-12-20 17:34:26 -08:00
Aleksander 955464e172 http.send: fix interquery cache bug (#5361)
This fixes the issue where concurrent requests with identical cache keys
cause the interquery cache size usage counter to become invalid.

Fixes #5359.

Signed-off-by: Aleksander <Alekken@live.no>
2022-11-24 10:13:34 +01:00
Anders Eknert 95708108f3 linters: add unconvert (#5318)
Got a few warnings from my IDE about redundant type conversions,
so I decided to look into it. Added the unconvert linter to our
checks, and fixed the violations. Added two ignore comments as I
wasn't sure about whether they'd change the semantics of the code.

Signed-off-by: Anders Eknert <anders@eknert.com>
2022-10-27 13:35:39 +02:00
vinhph0906 1ef6dc293d fix InterQueryCache Insert only drop one when size over limit
Signed-off-by: vinhph0906 <vinhph0906@gmail.com>
2022-09-28 10:06:58 -07:00
Ashutosh Narkar 7f7562505a topdown/cache: Check cache value size during insert operation
Previously while inserting an item in the cache, there was no
check to see if the item's size exceeded the cache limit. This
would result in that item being added to cache which is incorrect
behavior.This change adds an item size check during a cache insert.

Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
2021-06-09 16:17:48 -07:00
Grant Shively a1d8381fc8 plugins: inter-query cache config discovery
InterQueryBuiltinCacheConfig now responds to the plugin manager's reconfigure event, which allows cache config to exist in discovery config. Previously, cache config would be ignored if it was only declared in discovery config.

Related to #2978.

Signed-off-by: Grant Shively <gshively@godaddy.com>
2021-01-05 16:09:58 -08:00
Ashutosh Narkar 7f65b04561 Add a new inter-query cache to cache responses across queries
This commit adds a new inter-query cache that built-in
functions can use to cache responses across queries.

The OPA config includes a new "caching" field that can be used
to set the size of the cache. By default there is no limit.

This change also updates `http.send` to optionally utilize the
inter-query cache.

Fixes #1753

Signed-off-by: Ashutosh Narkar <anarkar4387@gmail.com>
2020-08-06 14:33:18 -07:00