Pushdown Prometheus Predicates #8742 #25957

jirislav · 2025-06-09T00:37:43Z

Description

Implements pushdown for Prometheus predicates, as requested in #8742.

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

## Prometheus Connector
* Support predicate pushdown for equality, non-equality, LIKE & IN operators. (#8742)

cla-bot · 2025-06-09T00:37:47Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

cla-bot · 2025-06-09T08:04:27Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

cla-bot · 2025-06-09T10:12:22Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

plugin/trino-prometheus/src/main/java/io/trino/plugin/prometheus/expression/RewriteIn.java

plugin/trino-prometheus/src/test/java/io/trino/plugin/prometheus/LoggedRequest.java

cla-bot · 2025-06-09T13:24:46Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

jirislav · 2025-06-09T13:27:25Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

I have the CLA signed and sent like 6 hours ago, not sure how long it usually takes to propagate to the related failing CI job.

To inspect compilation warnings, see https://github.com/trinodb/trino/actions/runs/15529824872/job/43716197311?pr=25957#step:5:149.

jirislav · 2025-06-24T19:05:54Z

@ebyhr could you please review & merge? I see you made recent changes to Prometheus Connector as well.

ebyhr

Just skimmed.

ebyhr · 2025-06-24T21:44:16Z

...in/trino-prometheus/src/main/java/io/trino/plugin/prometheus/expression/RewriteEqAndNeq.java

@@ -79,6 +79,6 @@ public Optional<ImmutableList<LabelFilterExpression>> rewrite(Call call, Capture
                labelName,
                labelValue,
                false,
-                call.getFunctionName() == NOT_EQUAL_OPERATOR_FUNCTION_NAME)));
+                call.getFunctionName().equals(NOT_EQUAL_OPERATOR_FUNCTION_NAME))));


This change should go to the 1st commit.

ebyhr · 2025-06-24T21:44:21Z

plugin/trino-prometheus/src/main/java/io/trino/plugin/prometheus/expression/RewriteHelpers.java

@@ -53,13 +53,13 @@ public static String extractLabelName(Captures captures, Capture<Call> captureKe
        // This method assumes that
        List<ConnectorExpression> arguments = captures.get(captureKey).getArguments();
        Object labelNameObject = ((Constant) arguments.getLast()).getValue();
-        if (!(labelNameObject instanceof Slice)) {
+        if (!(labelNameObject instanceof Slice slice)) {


This change should go to the 1st commit.

ebyhr · 2025-06-24T21:44:23Z

plugin/trino-prometheus/src/main/java/io/trino/plugin/prometheus/expression/RewriteHelpers.java

            if (labelNameObject == null) {
                throw new IllegalArgumentException("Label name cannot be null");
            }
            throw new IllegalArgumentException("Expected label name to be a Slice, but got: " + labelNameObject.getClass().getSimpleName());
        }
-        return ((Slice) labelNameObject).toStringUtf8();
+        return slice.toStringUtf8();


This change should go to the 1st commit.

ebyhr · 2025-06-24T21:45:13Z

plugin/trino-prometheus/src/main/java/io/trino/plugin/prometheus/PrometheusTableHandle.java

@@ -23,14 +25,15 @@

 import static java.util.Objects.requireNonNull;

-public record PrometheusTableHandle(String schemaName, String tableName, Optional<TupleDomain<ColumnHandle>> predicate)
+public record PrometheusTableHandle(String schemaName, String tableName, Optional<TupleDomain<ColumnHandle>> predicate, ImmutableList<LabelFilterExpression> expressions)


Suggested change

public record PrometheusTableHandle(String schemaName, String tableName, Optional<TupleDomain<ColumnHandle>> predicate, ImmutableList<LabelFilterExpression> expressions)

public record PrometheusTableHandle(String schemaName, String tableName, Optional<TupleDomain<ColumnHandle>> predicate, List<LabelFilterExpression> expressions)

ebyhr · 2025-06-24T21:45:36Z

plugin/trino-prometheus/src/main/java/io/trino/plugin/prometheus/PrometheusTableHandle.java

        implements ConnectorTableHandle
 {
    public PrometheusTableHandle
    {
        requireNonNull(schemaName, "schemaName is null");
        requireNonNull(tableName, "tableName is null");
        requireNonNull(predicate, "predicate is null");
+        requireNonNull(expressions, "expressions is null");


Remove requireNonNull and add ImmutableList.copyOf.

ebyhr · 2025-06-24T21:58:47Z

plugin/trino-prometheus/src/test/java/io/trino/plugin/prometheus/TestPrometheusIntegration.java

+            assertThat(request.extractPromQL()).isEqualTo("up{job=\"prometheus\"}[1d]");
+        });
+
+        assertThat(results).hasSizeBetween(1, 2);


Why is the result size non-deterministic? Please leave a code comment.

ebyhr · 2025-06-24T21:59:19Z

plugin/trino-prometheus/src/test/java/io/trino/plugin/prometheus/TestPrometheusIntegration.java

+            assertThat(request.extractPromQL()).isEqualTo("up[1d]");
+        });
+
+        assertThat(results).hasSize(0);


There is isEmpty() method.

ebyhr · 2025-06-24T21:59:32Z

plugin/trino-prometheus/src/test/java/io/trino/plugin/prometheus/TestPrometheusTableHandle.java

+        return newTableHandle(schemaName, tableName, Optional.empty(), ImmutableList.of());
+    }
+
+    public static PrometheusTableHandle newTableHandle(String schemaName, String tableName, Optional<TupleDomain<ColumnHandle>> predicate, ImmutableList<LabelFilterExpression> expressions)


Suggested change

public static PrometheusTableHandle newTableHandle(String schemaName, String tableName, Optional<TupleDomain<ColumnHandle>> predicate, ImmutableList<LabelFilterExpression> expressions)

public static PrometheusTableHandle newTableHandle(String schemaName, String tableName, Optional<TupleDomain<ColumnHandle>> predicate, List<LabelFilterExpression> expressions)

ebyhr · 2025-06-24T22:00:08Z

plugin/trino-prometheus/src/test/java/io/trino/plugin/prometheus/TestPrometheusIntegration.java


-public class TestPrometheusIntegration
+@Execution(SAME_THREAD)


Please leave a code comment why specifying SAME_THREAD.

ebyhr · 2025-06-24T22:01:52Z

plugin/trino-prometheus/src/test/java/io/trino/plugin/prometheus/PrometheusServer.java

+        prometheusContainer.close();
+
+        if (proxyContainer != null) {
+            proxyContainer.close();
+        }
+
+        network.close();


You can use Guava's Closer.

chenjian2664 · 2025-06-25T06:56:54Z

plugin/trino-prometheus/src/test/java/io/trino/plugin/prometheus/PrometheusServer.java

    private static final int PROMETHEUS_PORT = 9090;
-    private static final String DEFAULT_VERSION = "v2.15.1";


Does this changes required?

chenjian2664 · 2025-06-25T07:11:23Z

plugin/trino-prometheus/src/main/java/io/trino/plugin/prometheus/expression/RewriteHelpers.java

+    public static final Predicate<Call> LABEL_MAP_CHECK_CORRECT_TYPE = connectorExpression -> {
+        List<ConnectorExpression> arguments = connectorExpression.getArguments();
+
+        if (!(arguments.getFirst() instanceof Variable)) {


Add a check if arguments is empty or not

chenjian2664 · 2025-06-25T07:12:08Z

plugin/trino-prometheus/src/main/java/io/trino/plugin/prometheus/expression/RewriteHelpers.java

+        List<ConnectorExpression> arguments = captures.get(captureKey).getArguments();
+        Object labelNameObject = ((Constant) arguments.getLast()).getValue();
+        if (!(labelNameObject instanceof Slice)) {
+            if (labelNameObject == null) {


Should we perform this check earlier?

chenjian2664 · 2025-06-25T07:12:34Z

plugin/trino-prometheus/src/main/java/io/trino/plugin/prometheus/expression/RewriteHelpers.java

+        // Extract the label name from the call expression, which is actually the key of the labels map.
+        // This method assumes that
+        List<ConnectorExpression> arguments = captures.get(captureKey).getArguments();
+        Object labelNameObject = ((Constant) arguments.getLast()).getValue();


add arguments check if it is empty

github-actions bot added docs prometheus Prometheus connector labels Jun 9, 2025

jirislav mentioned this pull request Jun 9, 2025

Trino-Prometheus connector: Enable filter labels pushdown to the api/v1/query endpoint in addition to the timestamp and values filters #8742

Open

jirislav force-pushed the pushdown-promql-label-filter branch from 01a2274 to 62d8bd7 Compare June 9, 2025 08:04

jirislav force-pushed the pushdown-promql-label-filter branch from 62d8bd7 to dc6eaf4 Compare June 9, 2025 10:12

etolbakov reviewed Jun 9, 2025

View reviewed changes

plugin/trino-prometheus/src/main/java/io/trino/plugin/prometheus/expression/RewriteIn.java Outdated Show resolved Hide resolved

etolbakov reviewed Jun 9, 2025

View reviewed changes

plugin/trino-prometheus/src/test/java/io/trino/plugin/prometheus/LoggedRequest.java Outdated Show resolved Hide resolved

jirislav force-pushed the pushdown-promql-label-filter branch from dc6eaf4 to b2a6f1f Compare June 9, 2025 13:24

jirislav added 2 commits June 23, 2025 20:58

Push-Down Prometheus Predicates trinodb#8742

0353551

Resolve all compilation warnings for Prometheus Connector

6028e80

To inspect compilation warnings, see https://github.com/trinodb/trino/actions/runs/15529824872/job/43716197311?pr=25957#step:5:149.

jirislav force-pushed the pushdown-promql-label-filter branch from b2a6f1f to 6028e80 Compare June 23, 2025 18:58

cla-bot bot added the cla-signed label Jun 23, 2025

jirislav assigned ebyhr Jun 24, 2025

ebyhr removed their assignment Jun 24, 2025

ebyhr reviewed Jun 24, 2025

View reviewed changes

chenjian2664 reviewed Jun 25, 2025

View reviewed changes

	public record PrometheusTableHandle(String schemaName, String tableName, Optional<TupleDomain<ColumnHandle>> predicate, ImmutableList<LabelFilterExpression> expressions)
	public record PrometheusTableHandle(String schemaName, String tableName, Optional<TupleDomain<ColumnHandle>> predicate, List<LabelFilterExpression> expressions)

	public static PrometheusTableHandle newTableHandle(String schemaName, String tableName, Optional<TupleDomain<ColumnHandle>> predicate, ImmutableList<LabelFilterExpression> expressions)
	public static PrometheusTableHandle newTableHandle(String schemaName, String tableName, Optional<TupleDomain<ColumnHandle>> predicate, List<LabelFilterExpression> expressions)


		public class TestPrometheusIntegration
		@Execution(SAME_THREAD)

		private static final int PROMETHEUS_PORT = 9090;
		private static final String DEFAULT_VERSION = "v2.15.1";

Pushdown Prometheus Predicates #8742 #25957

Are you sure you want to change the base?

Pushdown Prometheus Predicates #8742 #25957

Conversation

jirislav commented Jun 9, 2025

Description

Additional context and related issues

Release notes

Uh oh!

cla-bot bot commented Jun 9, 2025

Uh oh!

cla-bot bot commented Jun 9, 2025

Uh oh!

cla-bot bot commented Jun 9, 2025

Uh oh!

Uh oh!

Uh oh!

cla-bot bot commented Jun 9, 2025

Uh oh!

jirislav commented Jun 9, 2025

Uh oh!

jirislav commented Jun 24, 2025

Uh oh!

ebyhr left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!