Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat][storage] Added the auto creation of Jaeger ILM/ISM policy for ES/OS #6604

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
76 changes: 76 additions & 0 deletions cmd/jaeger/config-elasticsearch-ilm.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
service:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need new configs? Let's change the main ones

extensions: [jaeger_storage, jaeger_query, healthcheckv2]
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [jaeger_storage_exporter]
telemetry:
resource:
service.name: jaeger
metrics:
level: detailed
address: 0.0.0.0:8888
logs:
level: debug
# TODO Initialize telemetry tracer once OTEL released new feature.
# https://github.com/open-telemetry/opentelemetry-collector/issues/10663

extensions:
healthcheckv2:
use_v2: true
http:

jaeger_query:
storage:
traces: some_storage
traces_archive: another_storage
ui:
config_file: ./cmd/jaeger/config-ui.json

jaeger_storage:
backends:
some_storage:
elasticsearch:
create_mappings: true
indices:
index_prefix: "jaeger-main"
spans:
date_layout: "2006-01-02"
rollover_frequency: "day"
shards: 5
replicas: 1
services:
date_layout: "2006-01-02"
rollover_frequency: "day"
shards: 5
replicas: 1
dependencies:
date_layout: "2006-01-02"
rollover_frequency: "day"
shards: 5
replicas: 1
sampling:
date_layout: "2006-01-02"
rollover_frequency: "day"
shards: 5
replicas: 1
use_ilm: true
use_aliases: true
another_storage:
elasticsearch:
indices:
index_prefix: "jaeger-archive"

receivers:
otlp:
protocols:
grpc:
http:

processors:
batch:

exporters:
jaeger_storage_exporter:
trace_storage: some_storage
2 changes: 2 additions & 0 deletions cmd/jaeger/config-elasticsearch.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@ extensions:
rollover_frequency: "day"
shards: 5
replicas: 1
use_ilm: true
use_aliases: true
another_storage:
elasticsearch:
server_urls:
Expand Down
76 changes: 76 additions & 0 deletions cmd/jaeger/config-opensearch-ilm.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
service:
extensions: [jaeger_storage, jaeger_query, healthcheckv2]
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [jaeger_storage_exporter]
telemetry:
resource:
service.name: jaeger
metrics:
level: detailed
address: 0.0.0.0:8888
logs:
level: debug
# TODO Initialize telemetry tracer once OTEL released new feature.
# https://github.com/open-telemetry/opentelemetry-collector/issues/10663

extensions:
healthcheckv2:
use_v2: true
http:

jaeger_query:
storage:
traces: some_storage
traces_archive: another_storage
ui:
config_file: ./cmd/jaeger/config-ui.json

jaeger_storage:
backends:
some_storage:
opensearch:
use_ilm: true
create_mappings: true
use_aliases: true
indices:
index_prefix: "jaeger-main"
spans:
date_layout: "2006-01-02"
rollover_frequency: "day"
shards: 5
replicas: 1
services:
date_layout: "2006-01-02"
rollover_frequency: "day"
shards: 5
replicas: 1
dependencies:
date_layout: "2006-01-02"
rollover_frequency: "day"
shards: 5
replicas: 1
sampling:
date_layout: "2006-01-02"
rollover_frequency: "day"
shards: 5
replicas: 1
another_storage:
opensearch:
indices:
index_prefix: "jaeger-archive"

receivers:
otlp:
protocols:
grpc:
http:

processors:
batch:

exporters:
jaeger_storage_exporter:
trace_storage: some_storage
2 changes: 2 additions & 0 deletions cmd/jaeger/internal/extension/jaegerstorage/extension.go
Original file line number Diff line number Diff line change
Expand Up @@ -155,12 +155,14 @@ func (s *storageExt) Start(_ context.Context, host component.Host) error {
s.telset.Logger,
)
case cfg.Elasticsearch != nil:
//nolint: contextcheck
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really don't know how can I pass context or do I need to supress the linter like this only?

factory, err = es.NewFactoryWithConfig(
*cfg.Elasticsearch,
scopedMetricsFactory(storageName, "elasticsearch", "tracestore"),
s.telset.Logger,
)
case cfg.Opensearch != nil:
//nolint: contextcheck
factory, err = es.NewFactoryWithConfig(
*cfg.Opensearch,
scopedMetricsFactory(storageName, "opensearch", "tracestore"),
Expand Down
Loading
Loading