Skip to content

Commit

Permalink
Amend allowed_attributes in network documentation (#683)
Browse files Browse the repository at this point in the history
* Amend allowed_attributes in network documentation

* Fix vale errors
  • Loading branch information
mariomac authored Mar 14, 2024
1 parent 4b26321 commit 28cb938
Show file tree
Hide file tree
Showing 3 changed files with 27 additions and 19 deletions.
15 changes: 9 additions & 6 deletions docs/sources/network/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,9 @@ To get started using Beyla networking metrics, consult the [quickstart setup doc

## Metric attributes

Network metrics provides a single **OpenTelemetry** metric `beyla.network.flow.bytes`, a counter of Number of bytes observed between two network endpoints, with the following attributes:
Network metrics provides a single **OpenTelemetry** metric `beyla.network.flow.bytes`, a counter of Number of bytes observed between two network endpoints, and can have the attributes in the following table.

By default, only the following attributes are reported: `k8s.src.owner.name`, `k8s.src.namespace`, `k8s.dst.owner.name`, `k8s.dst.namespace`, and `k8s.cluster.name`.

| Attribute name | Description |
|----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
Expand All @@ -36,8 +38,6 @@ Network metrics provides a single **OpenTelemetry** metric `beyla.network.flow.b
| `dst.address` | Destination IP address of Network flow |
| `src.name` | Name of Network flow source: Kubernetes name, host name, or IP address |
| `dst.name` | Name of Network flow destination: Kubernetes name, host name, or IP address |
| `src.namespace` | Namespace of Network flow source. Could be empty in non-Kubernetes flows |
| `dst.namespace` | Namespace of Network flow destination. Could be empty in non-Kubernetes flows |
| `src.cidr` | If the [`cidrs` configuration section]({{< relref "./config" >}}) is set, the CIDR that matches the source IP address |
| `dst.cidr` | If the [`cidrs` configuration section]({{< relref "./config" >}}) is set, the CIDR that matches the destination IP address |
| `k8s.src.namespace` | Kubernetes namespace of the source of the flow |
Expand All @@ -54,11 +54,13 @@ Network metrics provides a single **OpenTelemetry** metric `beyla.network.flow.b
| `k8s.dst.node.name` | Name of the destination Node |
| `k8s.cluster.name` | Name of the Kubernetes cluster. Beyla can auto-detect it on Google Cloud, Microsoft Azure, and Amazon Web Services. For other providers, set the `BEYLA_KUBE_CLUSTER_NAME` property |

### Allowed attributes
### How to specify reported attributes

If the metric with all the possible attributes is reported it might lead to a cardinality explosion, especially when including external traffic in the `src.address`/`dst.address` attributes.

If the metric with all the attributes is reported it might lead to a cardinality explosion, especially when including external traffic in the `src.address`/`dst.address` attributes.
You can specify which attributes are allowed in the Beyla configuration, to aggregate the metric by them.

You can specify which attributes are allowed in the Beyla configuration. Allowed attributes and aggregates the metrics by them. For example:
For example:

```yaml
network:
Expand All @@ -68,6 +70,7 @@ network:
- k8s.src.namespace
- k8s.dst.owner.name
- k8s.dst.namespace
- k8s.cluster.name
```
In this example, the bytes metric is the aggregated by the source and destination owners. This is, all the
Expand Down
12 changes: 6 additions & 6 deletions docs/sources/network/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,23 +43,23 @@ network:

Enables network metrics reporting in Beyla.

| YAML | Environment variable | Type | Default |
| -------------------- | ---------------------------------- | -------- | ------- |
| `allowed_attributes` | `BEYLA_NETWORK_ALLOWED_ATTRIBUTES` | []string | (empty) |
| YAML | Environment variable | Type | Default |
| -------------------- | ---------------------------------- | -------- | -------------------------------------------------------------------------------------------------------- |
| `allowed_attributes` | `BEYLA_NETWORK_ALLOWED_ATTRIBUTES` | []string | `k8s.src.owner.name`, `k8s.src.namespace`, `k8s.dst.owner.name`, `k8s.dst.namespace`, `k8s.cluster.name` |

Specifies which attributes are visible in the metrics.
Beyla aggregates the metrics by their common visible attributes.
For example, hiding the `k8s.src.name` and allowing `k8s.src.owner.name` would aggregate the metrics of all the pods under the same owner.

This property won't filter some meta-attributes such as `instance`, `job`, `service.instance.id`, `service_name`, `telemetry.sdk.*`, etc.

See the [network metrics documentation]({{< relref "./_index.md" >}}) for a detailed list of all the available attributes.

{{% admonition type="note" %}}
If left empty, Beyla reports all attributes; which might greatly increase the cardinality of your metrics.
Select carefully the reported attributes, as some attributes might greatly increase the cardinality of your metrics.
Setting this value to list only the attributes you really need is highly recommended.
{{% /admonition %}}

See the [network metrics documentation]({{< relref "./_index.md" >}}) for a detailed list of all the available attributes.

If you set this property via environment variable each entry must be separated by a comma, for example:

```sh
Expand Down
19 changes: 12 additions & 7 deletions docs/sources/network/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,8 +146,7 @@ Use `kubectl logs` to see network flow entries, for example:

```
network_flow: beyla.ip=172.18.0.2 iface= direction=255 src.address=10.244.0.4 dst.address=10.96.0.1
src.name=local-path-provisioner-7577fdbbfb-g6b7d src.namespace=local-path-storage
dst.name=kubernetes dst.namespace=default k8s.src.node.ip=172.18.0.2
src.name=local-path-provisioner-7577fdbbfb-g6b7d dst.name=kubernetes
k8s.src.node.name=kind-control-plane k8s.dst.namespace=default k8s.dst.name=kubernetes
k8s.dst.owner.type=Service k8s.src.namespace=local-path-storage
k8s.src.name=local-path-provisioner-7577fdbbfb-g6b7d k8s.src.type=Pod
Expand Down Expand Up @@ -209,14 +208,20 @@ Also Add `OTEL_EXPORTER_OTLP_ENDPOINT` and its value as an environment variable

## Select metrics attributes to reduce cardinality

Letting Beyla to include all the [attributes]({{< relref "./_index.md" >}}) in the reported metric might lead to
a [cardinality explosion](/blog/2022/02/15/what-are-cardinality-spikes-and-why-do-they-matter/) in
your metrics storage, especially if you are capturing external traffic and reporting their IP addresses in the
`src.address` or `dst.address` metric attribute.
Be default, Beyla includes the following [attributes]({{< relref "./_index.md" >}}) in the `beyla.network.flow.bytes` metric:

- `k8s.src.owner.name`
- `k8s.src.namespace`
- `k8s.dst.owner.name`
- `k8s.dst.namespace`
- `k8s.cluster.name`

Beyla only includes a subset of the available attributes to avoid leading to
a [cardinality explosion](/blog/2022/02/15/what-are-cardinality-spikes-and-why-do-they-matter/) in
the metrics storage, especially if some attributes like `src.address` or `dst.address` capture the IP addresses of the external traffic.

The `allowed_attributes` YAML subsection under `network` (or the `BEYLA_NETWORK_ALLOWED_ATTRIBUTES` environment variable)
lets you selecting the attributes to report:
lets to select the attributes to report:

```yaml
network:
Expand Down

0 comments on commit 28cb938

Please sign in to comment.