Add Prometheus metrics to the status updates

Prometheus metrics can give much of insight into OPA's health.
Run-time metrics are a natural part of the application state
so having them in status update seems like a right change
that can help server understand what's going in with the OPA
instance.

The commit also encapsulates all prometheus-related code in one package
and abstracts it with generic interface so that it would be possible
to add other metrics providers

Addresses #1606

Signed-off-by: Stan Lagun <stan@styra.com>
This commit is contained in:
Stan Lagun
2019-08-05 17:45:10 -07:00
committed by Torin Sandall
parent ae99b40b33
commit 69fbaacfde
15 changed files with 460 additions and 160 deletions
+16
View File
@@ -52,6 +52,9 @@ status:
service: acmecorp
default_decision: /http/example/authz/allow
metrics_provider:
name: prometheus
```
## Environment Variable Substitution
@@ -343,6 +346,7 @@ server provenance, etc.
| --- | --- | --- | --- |
| `status.service` | `string` | Yes | Name of service to use to contact remote server. |
| `status.partition_name` | `string` | No | Path segment to include in status updates. |
| `status.include_metrics` | `boolean` | (default: `false`) | Include Prometheus metrics in status updates. |
## Decision Logs
@@ -367,3 +371,15 @@ server provenance, etc.
| `discovery.decision` | `string` | No (default: value of `discovery.name` configuration field) | Name of the OPA query that will be used to calculate the configuration |
| `discovery.polling.min_delay_seconds` | `int64` | No (default: `60`) | Minimum amount of time to wait between configuration downloads. |
| `discovery.polling.max_delay_seconds` | `int64` | No (default: `120`) | Maximum amount of time to wait between configuration downloads. |
## Metrics provider
| Field | Type | Required | Description |
| --- | --- | --- | --- |
| `metrics_provider.name` | `string` | No | Name of the metrics provider to use. |
| `metrics_provider.config` | `object` | No | Provider-specific configuration (not used as of now). |
Available metrics providers:
* `prometheus` (default)
* (empty string): do not collect metrics
+71
View File
@@ -46,6 +46,75 @@ on the agent, updates will be sent to `/status`.
"last_successful_download": "2018-01-01T00:00:00.000Z",
"last_successful_activation": "2018-01-01T00:00:00.000Z"
}
},
"metrics": {
"prometheus": [
{
"help": "A summary of the GC invocation durations.",
"name": "go_gc_duration_seconds",
"type": 2,
"metric": [
{
"summary": {
"quantile": [
{
"quantile": 0,
"value": 0.000044358
},
{
"quantile": 0.25,
"value": 0.000045003
},
{
"quantile": 0.5,
"value": 0.000049726
},
{
"quantile": 0.75,
"value": 0.000219553
},
{
"quantile": 1,
"value": 0.000219553
}
],
"sample_count": 4,
"sample_sum": 0.00035864
}
}
]
},
{
"help": "Number of goroutines that currently exist.",
"name": "go_goroutines",
"type": 1,
"metric": [
{
"gauge": {
"value": 11
}
}
]
},
{
"help": "Information about the Go environment.",
"name": "go_info",
"type": 1,
"metric": [
{
"gauge": {
"value": 1
},
"label": [
{
"name": "version",
"value": "go1.12.7"
}
]
}
]
}
]
}
}
```
@@ -68,6 +137,8 @@ Status updates contain the following fields:
| `discovery.active_revision` | `string` | Opaque revision identifier of the last successful discovery activation. |
| `discovery.last_successful_download` | `string` | RFC3339 timestamp of last successful discovery bundle download. |
| `discovery.last_successful_activation` | `string` | RFC3339 timestamp of last successful discovery bundle activation. |
| `metrics` | `object` | Application metrics. Optional, single key object. |
| `metrics[provider_name]` | JSON (`interface{}`) | Metrics in provider-dependent format. |
If the bundle download or activation failed, the status update will contain
the following additional fields.