Static labels disappear when being copied from one processor to another in a chain

[Elecon-rou]

Bug Report

Describe the bug

When multiple labels metric processors are chained, upsert labels created by an earlier processor disappear.

The processor creates a new CMetrics context and copies the previous one with cmt_cat():

https://github.com/fluent/fluent-bit/blob/master/plugins/processor_labels/labels.c

cmt_cat() does not copy CMetrics static labels:

https://github.com/fluent/fluent-bit/blob/master/lib/cmetrics/src/cmt_cat.c

To Reproduce

test_metric{original="value"} 1

pipeline:
  inputs:
    - name: prometheus_scrape
      tag: test
      host: 127.0.0.1
      port: 9090

      processors:
        metrics:
          - name: labels
            upsert: "first one"
          - name: labels
            upsert: "second two"

  outputs:
    - name: prometheus_exporter
      match: test
      port: 2021

Expected

test_metric{original="value",first="one",second="two"} 1

Actual

test_metric{original="value",second="two"} 1

Your Environment

• Version: 5.1-debug
• Plugins: prometheus_scrape, labels, prometheus_exporter

[cosmo0920]

This can be fixed by fluent/cmetrics#302.

[cosmo0920]

In Fluent Bit, we also have a canary PR: #12420