Using grep filter as conditional log processor ignores any condition clauses

[john-forrest]

Bug Report

Setup using a grep filter as a log processor and give it a condition. Seems the condition is completely ignored - there are no obvious errors but fluent-bit gives the output as if it is not there.

To Reproduce

Run with the following configuration file (adapted from some of the documentation examples):

pipeline:
  inputs:
    - name: dummy
      dummy: '{"endpoint":"localhost", "value":"something"},
              {"endpoint":"localhost2", "value":"something"},
              {"endpoint":"farhost", "value":"nothing"}'
      tag: dummy
      processors:
        logs:
          - name: grep
            logical_op: and
            regex:
              - value something
            condition:
              op: and
              rules:
                - field: "$endpoint"
                  op: eq
                  value: "localhost2"

  outputs:
    - name: stdout
      match: '*'

OK this is not explicitly documented behaviour, but I would expect it to behave so that: if the condition
matched (endpoiint==localhost2) then the grep operated so value needs to equal something, otherwise
the grep to be ignored. Thus I would expect matches on all lines but for different reasons. What I
actually see are lines like:

[0] dummy: [[1767349589.147330566, {}], {"endpoint"=>"localhost", "value"=>"something"}]
[1] dummy: [[1767349589.147330566, {}], {"endpoint"=>"localhost2", "value"=>"something"}]
[0] dummy: [[1767349590.147556274, {}], {"endpoint"=>"localhost", "value"=>"something"}]
[1] dummy: [[1767349590.147556274, {}], {"endpoint"=>"localhost2", "value"=>"som

It is matching the first two lines where the value is "something" and seemingly ignoring endpoint.

Your Environment

• Version: 4.2.0
• I am actually running "docker run --rm -v .:/fluent-bit/etc-extra fluent/fluent-bit:4.2.0-amd64 --config=/fluent-bit/etc-extra/grep-processor-with-cond.yaml" where grep-processor-with-cond.yaml is the config file above.

Additional context

What I am actually trying to do is to reduce the size of the logs we generate because for some generated
logs (I can tell these from a regular expression on one of the fields - let's say "foo .bar.) but we always
to keep summary fields even in this category (again we can tell from another field, say "hum .bug.).
Effectively we want to keep the record if ("NOT foo =~ .bar. OR hum =~ .bug.) the NOT bit is important
because officially you can't do NOTs with regex, so on face value this needs to be an exclude or similar
but not clear if we can do this combined statement with a normal grep.