Skip to content

Commit 377fe47

Browse files
thampiotrptodevwildumdehaansaclayton-cornell
authored
Cherry pick for rc1 (#2059)
* update changelog version * Prototype targets.merge function (array.combine_maps) (#1826) * Prototype map.inner_join function * Rename to "targets.merge", remove unnecessary params * Add failure tests * Delete incorrect comment * rename targets.merge to array.combine_maps, mark it as experimental, make is more permissive and add tests * update changelog * Update CHANGELOG.md Co-authored-by: Piotr <[email protected]> * Update docs/sources/reference/stdlib/array.md Co-authored-by: Piotr <[email protected]> * Update docs/sources/reference/stdlib/array.md Co-authored-by: Piotr <[email protected]> * Update docs/sources/reference/stdlib/array.md Co-authored-by: Piotr <[email protected]> * rename mapCombine to combineMaps * document panic * add equal test * add more tests * Update syntax/internal/stdlib/stdlib.go Co-authored-by: Piotr <[email protected]> * Update syntax/internal/stdlib/stdlib.go Co-authored-by: Piotr <[email protected]> * add examples in doc * fix error propagation * remove value nul on len function * refactor code into a traversal function * update doc to avoid modifying the experimental shared doc --------- Co-authored-by: William Dumont <[email protected]> Co-authored-by: Piotr <[email protected]> * Implement an initial version of the support bundle in Alloy (#2009) * Implement an initial version of the support bundle in Alloy * Add documentation for support bundle * Update changelog * Update docs/sources/troubleshoot/support_bundle.md Co-authored-by: Clayton Cornell <[email protected]> * Update docs/sources/troubleshoot/support_bundle.md Co-authored-by: Clayton Cornell <[email protected]> * Update docs/sources/troubleshoot/support_bundle.md Co-authored-by: Clayton Cornell <[email protected]> * Update docs/sources/troubleshoot/support_bundle.md Co-authored-by: Clayton Cornell <[email protected]> * Initial PR feedback * Rewrite http service to use logging library internal to alloy * Revert accidental commit of e2e test changes * Fix comment on exported function * Clean up added host variable that is no longer used * Refactor usage of logger in http service * Update internal/service/http/http.go Co-authored-by: Piotr <[email protected]> * implement PR feedback * Hide support bundle behind public preview stability level * Update docs based on feedback * Update docs/sources/troubleshoot/support_bundle.md Co-authored-by: Clayton Cornell <[email protected]> * Update docs/sources/troubleshoot/support_bundle.md Co-authored-by: Clayton Cornell <[email protected]> * Update docs/sources/troubleshoot/support_bundle.md Co-authored-by: Clayton Cornell <[email protected]> * Update docs/sources/troubleshoot/support_bundle.md Co-authored-by: Clayton Cornell <[email protected]> * Update docs/sources/troubleshoot/support_bundle.md Co-authored-by: Clayton Cornell <[email protected]> * Update docs/sources/troubleshoot/support_bundle.md Co-authored-by: Clayton Cornell <[email protected]> * More PR feedback in docs * Fix race condition in logger * Add a note about backward-compatibility exception --------- Co-authored-by: Clayton Cornell <[email protected]> Co-authored-by: Piotr <[email protected]> --------- Co-authored-by: Paulin Todev <[email protected]> Co-authored-by: William Dumont <[email protected]> Co-authored-by: Sam DeHaan <[email protected]> Co-authored-by: Clayton Cornell <[email protected]>
1 parent 769e7c3 commit 377fe47

File tree

19 files changed

+829
-33
lines changed

19 files changed

+829
-33
lines changed

CHANGELOG.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ This document contains a historical list of changes between releases. Only
77
changes that impact end-user behavior are listed; changes to documentation or
88
internal API changes are not present.
99

10-
v1.5.0-rc.0
10+
v1.5.0-rc.1
1111
-----------------
1212

1313
### Breaking changes
@@ -25,7 +25,10 @@ v1.5.0-rc.0
2525

2626
### Features
2727

28+
- Add support bundle generation via the API endpoint /-/support (@dehaansa)
29+
2830
- Add the function `path_join` to the stdlib. (@wildum)
31+
2932
- Add `pyroscope.receive_http` component to receive and forward Pyroscope profiles (@marcsanmi)
3033

3134
- Add support to `loki.source.syslog` for the RFC3164 format ("BSD syslog"). (@sushain97)
@@ -37,6 +40,8 @@ v1.5.0-rc.0
3740
- (_Experimental_) Add a `prometheus.write.queue` component to add an alternative to `prometheus.remote_write`
3841
which allowing the writing of metrics to a prometheus endpoint. (@mattdurham)
3942

43+
- (_Experimental_) Add the `arrary.combine_maps` function to the stdlib. (@ptodev, @wildum)
44+
4045
### Enhancements
4146

4247
- The `mimir.rules.kubernetes` component now supports adding extra label matchers

docs/sources/reference/cli/run.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,7 @@ The following flags are supported:
4242
* `--server.http.ui-path-prefix`: Base path where the UI is exposed (default `/`).
4343
* `--storage.path`: Base directory where components can store data (default `data-alloy/`).
4444
* `--disable-reporting`: Disable [data collection][] (default `false`).
45+
* `--disable-support-bundle`: Disable [support bundle][] endpoint (default `false`).
4546
* `--cluster.enabled`: Start {{< param "PRODUCT_NAME" >}} in clustered mode (default `false`).
4647
* `--cluster.node-name`: The name to use for this node (defaults to the environment's hostname).
4748
* `--cluster.join-addresses`: Comma-separated list of addresses to join the cluster at (default `""`). Mutually exclusive with `--cluster.discover-peers`.
@@ -178,6 +179,7 @@ Refer to [alloy convert][] for more details on how `extra-args` work.
178179
[go-discover]: https://github.com/hashicorp/go-discover
179180
[in-memory HTTP traffic]: ../../../get-started/component_controller/#in-memory-traffic
180181
[data collection]: ../../../data-collection/
182+
[support bundle]: ../../../troubleshoot/support_bundle
181183
[components]: ../../get-started/components/
182184
[component controller]: ../../../get-started/component_controller/
183185
[UI]: ../../../troubleshoot/debug/#clustering-page

docs/sources/reference/stdlib/array.md

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,3 +32,54 @@ Elements within the list can be any type.
3232
> array.concat([[1, 2], [3, 4]], [[5, 6]])
3333
[[1, 2], [3, 4], [5, 6]]
3434
```
35+
36+
## array.combine_maps
37+
38+
> **EXPERIMENTAL**: This is an [experimental][] feature. Experimental
39+
> features are subject to frequent breaking changes, and may be removed with
40+
> no equivalent replacement. The `stability.level` flag must be set to `experimental`
41+
> to use the feature.
42+
43+
The `array.combine_maps` function allows you to join two arrays of maps if certain keys have matching values in both maps. It's particularly useful when combining labels of targets coming from different `prometheus.discovery.*` or `prometheus.exporter.*` components.
44+
It takes three arguments:
45+
46+
* The first two arguments are a of type `list(map(string))`. The keys of the map are strings.
47+
The value for each key could be of any Alloy type such as a `string`, `integer`, `map`, or a `capsule`.
48+
* The third input is an `array` containing strings. The strings are the keys whose value has to match for maps to be combined.
49+
50+
The maps that don't contain all the keys provided in the third argument will be discarded. When maps are combined and both contain the same keys, the last value from the second argument will be used.
51+
52+
Pseudo function code:
53+
```
54+
for every map in arg1:
55+
for every map in arg2:
56+
if the condition key matches in both:
57+
merge maps and add to result
58+
```
59+
60+
### Examples
61+
62+
```alloy
63+
> array.combine_maps([{"instance"="1.1.1.1", "team"="A"}], [{"instance"="1.1.1.1", "cluster"="prod"}], ["instance"])
64+
[{"instance"="1.1.1.1", "team"="A", "cluster"="prod"}]
65+
66+
// Second map overrides the team in the first map
67+
> array.combine_maps([{"instance"="1.1.1.1", "team"="A"}], [{"instance"="1.1.1.1", "team"="B"}], ["instance"])
68+
[{"instance"="1.1.1.1", "team"="B"}]
69+
70+
// If multiple maps from the first argument match with multiple maps from the second argument, different combinations will be created.
71+
> array.combine_maps([{"instance"="1.1.1.1", "team"="A"}, {"instance"="1.1.1.1", "team"="B"}], [{"instance"="1.1.1.1", "cluster"="prod"}, {"instance"="1.1.1.1", "cluster"="ops"}], ["instance"])
72+
[{"instance"="1.1.1.1", "team"="A", "cluster"="prod"}, {"instance"="1.1.1.1", "team"="A", "cluster"="ops"}, {"instance"="1.1.1.1", "team"="B", "cluster"="prod"}, {"instance"="1.1.1.1", "team"="B", "cluster"="ops"}]
73+
```
74+
75+
Examples using discovery and exporter components:
76+
```alloy
77+
> array.combine_maps(discovery.kubernetes.k8s_pods.targets, prometheus.exporter.postgres, ["instance"])
78+
79+
> array.combine_maps(prometheus.exporter.redis.default.targets, [{"instance"="1.1.1.1", "testLabelKey" = "testLabelVal"}], ["instance"])
80+
```
81+
82+
You can find more examples in the [tests][].
83+
84+
[tests]: https://github.com/grafana/alloy/blob/main/syntax/vm/vm_stdlib_test.go
85+
[experimental]: https://grafana.com/docs/release-life-cycle/
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
---
2+
canonical: https://grafana.com/docs/alloy/latest/troubleshoot/support_bundle/
3+
description: Learn how to generate a support bundle
4+
title: Generate a support bundle
5+
menuTitle: Generate a support bundle
6+
weight: 300
7+
---
8+
9+
<span class="badge docs-labels__stage docs-labels__item">Public preview</span>
10+
11+
# Generate a support bundle
12+
13+
{{< docs/public-preview product="Generate support bundle" >}}
14+
15+
The `/-/support?duration=N` endpoint returns a support bundle, a zip file that contains information
16+
about a running {{< param "PRODUCT_NAME" >}} instance, and can be used as a baseline of information when trying
17+
to debug an issue.
18+
19+
This feature is not covered by our [backward-compatibility][backward-compatibility] guarantees.
20+
21+
{{< admonition type="note" >}}
22+
This endpoint is enabled by default, but may be disabled using the `--disable-support-bundle` runtime flag.
23+
{{< /admonition >}}
24+
25+
The duration parameter is optional, must be less than or equal to the
26+
configured HTTP server write timeout, and if not provided, defaults to it.
27+
The endpoint is only exposed to the {{< param "PRODUCT_NAME" >}} HTTP server listen address, which
28+
defaults to `localhost:12345`.
29+
30+
The support bundle contains all information in plain text, so you can
31+
inspect it before sharing to verify that no sensitive information has leaked.
32+
33+
In addition, you can inspect the [supportbundle implementation](https://github.com/grafana/alloy/tree/internal/service/http/supportbundle.go)
34+
to verify the code used to generate these bundles.
35+
36+
A support bundle contains the following data:
37+
* `alloy-components.json` contains information about the [components][components] running on this {{< param "PRODUCT_NAME" >}} instance, generated by the
38+
`/api/v0/web/components` endpoint.
39+
* `alloy-logs.txt` contains the logs during the bundle generation.
40+
* `alloy-metadata.yaml` contains the {{< param "PRODUCT_NAME" >}} build version and the installation's operating system, architecture, and uptime.
41+
* `alloy-metrics.txt` contains a snapshot of the internal metrics for {{< param "PRODUCT_NAME" >}}.
42+
* `alloy-peers.json` contains information about the identified cluster peers of this {{< param "PRODUCT_NAME" >}} instance, generated by the
43+
`/api/v0/web/peers` endpoint.
44+
* `alloy-runtime-flags.txt` contains the values of the runtime flags available in {{< param "PRODUCT_NAME" >}}.
45+
* The `pprof/` directory contains Go runtime profiling data (CPU, heap, goroutine, mutex, block profiles) as exported by the pprof package.
46+
Refer to the [profile][profile] documentation for more details on how to use this information.
47+
48+
[profile]: ../profile
49+
[components]: ../../get-started/components/
50+
[alloy-repo]: https://github.com/grafana/alloy/issues
51+
[backward-compatibility]: ../../introduction/backward-compatibility

go.mod

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -842,6 +842,8 @@ require (
842842
go.opentelemetry.io/otel/exporters/stdout/stdoutlog v0.7.0 // indirect
843843
)
844844

845+
require github.com/mackerelio/go-osstat v0.2.5
846+
845847
// NOTE: replace directives below must always be *temporary*.
846848
//
847849
// Adding a replace directive to change a module to a fork of a module will

go.sum

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1702,6 +1702,8 @@ github.com/lufia/plan9stats v0.0.0-20220913051719-115f729f3c8c h1:VtwQ41oftZwlMn
17021702
github.com/lufia/plan9stats v0.0.0-20220913051719-115f729f3c8c/go.mod h1:JKx41uQRwqlTZabZc+kILPrO/3jlKnQ2Z8b7YiVw5cE=
17031703
github.com/lyft/protoc-gen-validate v0.0.0-20180911180927-64fcb82c878e/go.mod h1:XbGvPuh87YZc5TdIa2/I4pLk0QoUACkjt2znoq26NVQ=
17041704
github.com/lyft/protoc-gen-validate v0.0.13/go.mod h1:XbGvPuh87YZc5TdIa2/I4pLk0QoUACkjt2znoq26NVQ=
1705+
github.com/mackerelio/go-osstat v0.2.5 h1:+MqTbZUhoIt4m8qzkVoXUJg1EuifwlAJSk4Yl2GXh+o=
1706+
github.com/mackerelio/go-osstat v0.2.5/go.mod h1:atxwWF+POUZcdtR1wnsUcQxTytoHG4uhl2AKKzrOajY=
17051707
github.com/magefile/mage v1.15.0 h1:BvGheCMAsG3bWUDbZ8AyXXpCNwU9u5CB6sM+HNb9HYg=
17061708
github.com/magefile/mage v1.15.0/go.mod h1:z5UZb/iS3GoOSn0JgWuiw7dxlurVYTu+/jHXqQg881A=
17071709
github.com/magiconair/properties v1.8.1/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=

internal/alloycli/cmd_run.go

Lines changed: 20 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ import (
2323
"github.com/grafana/ckit/peer"
2424
"github.com/prometheus/client_golang/prometheus"
2525
"github.com/spf13/cobra"
26+
"github.com/spf13/pflag"
2627
"go.opentelemetry.io/otel"
2728
"golang.org/x/exp/maps"
2829

@@ -64,6 +65,7 @@ func runCommand() *cobra.Command {
6465
clusterAdvInterfaces: advertise.DefaultInterfaces,
6566
clusterMaxJoinPeers: 5,
6667
clusterRejoinInterval: 60 * time.Second,
68+
disableSupportBundle: false,
6769
}
6870

6971
cmd := &cobra.Command{
@@ -100,7 +102,7 @@ depending on the nature of the reload error.
100102
SilenceUsage: true,
101103

102104
RunE: func(cmd *cobra.Command, args []string) error {
103-
return r.Run(args[0])
105+
return r.Run(cmd, args[0])
104106
},
105107
}
106108

@@ -111,6 +113,8 @@ depending on the nature of the reload error.
111113
cmd.Flags().StringVar(&r.uiPrefix, "server.http.ui-path-prefix", r.uiPrefix, "Prefix to serve the HTTP UI at")
112114
cmd.Flags().
113115
BoolVar(&r.enablePprof, "server.http.enable-pprof", r.enablePprof, "Enable /debug/pprof profiling endpoints.")
116+
cmd.Flags().
117+
BoolVar(&r.disableSupportBundle, "server.http.disable-support-bundle", r.disableSupportBundle, "Disable /-/support support bundle retrieval.")
114118

115119
// Cluster flags
116120
cmd.Flags().
@@ -184,9 +188,10 @@ type alloyRun struct {
184188
configBypassConversionErrors bool
185189
configExtraArgs string
186190
enableCommunityComps bool
191+
disableSupportBundle bool
187192
}
188193

189-
func (fr *alloyRun) Run(configPath string) error {
194+
func (fr *alloyRun) Run(cmd *cobra.Command, configPath string) error {
190195
var wg sync.WaitGroup
191196
defer wg.Wait()
192197

@@ -275,8 +280,15 @@ func (fr *alloyRun) Run(configPath string) error {
275280
return err
276281
}
277282

283+
runtimeFlags := []string{}
284+
if !fr.disableSupportBundle {
285+
cmd.Flags().VisitAll(func(f *pflag.Flag) {
286+
runtimeFlags = append(runtimeFlags, fmt.Sprintf("%s=%s", f.Name, f.Value.String()))
287+
})
288+
}
289+
278290
httpService := httpservice.New(httpservice.Options{
279-
Logger: log.With(l, "service", "http"),
291+
Logger: l,
280292
Tracer: t,
281293
Gatherer: prometheus.DefaultGatherer,
282294

@@ -286,6 +298,11 @@ func (fr *alloyRun) Run(configPath string) error {
286298
HTTPListenAddr: fr.httpListenAddr,
287299
MemoryListenAddr: fr.inMemoryAddr,
288300
EnablePProf: fr.enablePprof,
301+
MinStability: fr.minStability,
302+
BundleContext: httpservice.SupportBundleContext{
303+
RuntimeFlags: runtimeFlags,
304+
DisableSupportBundle: fr.disableSupportBundle,
305+
},
289306
})
290307

291308
remoteCfgService, err := remotecfgservice.New(remotecfgservice.Options{

internal/runtime/internal/controller/component_references.go

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ import (
55
"strings"
66

77
"github.com/go-kit/log"
8+
"github.com/grafana/alloy/internal/featuregate"
89
"github.com/grafana/alloy/internal/runtime/internal/dag"
910
"github.com/grafana/alloy/internal/runtime/logging/level"
1011
"github.com/grafana/alloy/syntax/ast"
@@ -18,6 +19,17 @@ import (
1819
// will be (field_a, field_b, field_c).
1920
type Traversal []*ast.Ident
2021

22+
// String returns a dot-separated string representation of the field names in the traversal.
23+
// For example, a traversal of fields [field_a, field_b, field_c] returns "field_a.field_b.field_c".
24+
// Returns an empty string if the traversal contains no fields.
25+
func (t Traversal) String() string {
26+
var fieldNames []string
27+
for _, field := range t {
28+
fieldNames = append(fieldNames, field.Name)
29+
}
30+
return strings.Join(fieldNames, ".")
31+
}
32+
2133
// Reference describes an Alloy expression reference to a BlockNode.
2234
type Reference struct {
2335
Target BlockNode // BlockNode being referenced
@@ -29,7 +41,7 @@ type Reference struct {
2941

3042
// ComponentReferences returns the list of references a component is making to
3143
// other components.
32-
func ComponentReferences(cn dag.Node, g *dag.Graph, l log.Logger, scope *vm.Scope) ([]Reference, diag.Diagnostics) {
44+
func ComponentReferences(cn dag.Node, g *dag.Graph, l log.Logger, scope *vm.Scope, minStability featuregate.Stability) ([]Reference, diag.Diagnostics) {
3345
var (
3446
traversals []Traversal
3547

@@ -63,6 +75,16 @@ func ComponentReferences(cn dag.Node, g *dag.Graph, l log.Logger, scope *vm.Scop
6375
refs = append(refs, ref)
6476
} else if scope.IsStdlibDeprecated(t[0].Name) {
6577
level.Warn(l).Log("msg", "this stdlib function is deprecated; please refer to the documentation for updated usage and alternatives", "function", t[0].Name)
78+
} else if funcName := t.String(); scope.IsStdlibExperimental(funcName) {
79+
if err := featuregate.CheckAllowed(featuregate.StabilityExperimental, minStability, funcName); err != nil {
80+
diags = append(diags, diag.Diagnostic{
81+
Severity: diag.SeverityLevelError,
82+
Message: err.Error(),
83+
StartPos: ast.StartPos(t[0]).Position(),
84+
EndPos: ast.StartPos(t[len(t)-1]).Position(),
85+
})
86+
continue
87+
}
6688
}
6789
}
6890

internal/runtime/internal/controller/loader.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -615,7 +615,7 @@ func (l *Loader) wireGraphEdges(g *dag.Graph) diag.Diagnostics {
615615

616616
// Finally, wire component references.
617617
l.cache.mut.RLock()
618-
refs, nodeDiags := ComponentReferences(n, g, l.log, l.cache.scope)
618+
refs, nodeDiags := ComponentReferences(n, g, l.log, l.cache.scope, l.globals.MinStability)
619619
l.cache.mut.RUnlock()
620620
for _, ref := range refs {
621621
g.AddEdge(dag.Edge{From: n, To: ref.Target})

0 commit comments

Comments
 (0)