Skip to content

Add SHOW RECEIVERS support - #17869

Open
Caideyipi wants to merge 53 commits into
masterfrom
feat/show-receivers
Open

Add SHOW RECEIVERS support#17869
Caideyipi wants to merge 53 commits into
masterfrom
feat/show-receivers

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Summary

  • add tree-model SHOW RECEIVERS and table-model information_schema.receivers support
  • track active pipe receiver sessions, sender endpoints, pipe IDs, handshake/transfer timestamps, and request counts
  • wire DataNode/ConfigNode thrift and air-gap receiver runtime recording

Tests

  • mvn "-Ddevelocity.off=true" -pl iotdb-core/node-commons -Dtest=PipeReceiverRuntimeRegistryTest test
  • mvn "-Ddevelocity.off=true" -DskipTests compile -pl iotdb-core/antlr,iotdb-core/relational-grammar,iotdb-core/node-commons,iotdb-core/confignode
  • mvn "-Ddevelocity.off=true" -DskipTests spotless:check -pl iotdb-core/node-commons,iotdb-core/datanode,iotdb-core/confignode,iotdb-core/relational-grammar,iotdb-core/antlr
  • git diff --check

Notes

  • DataNode focused compile is still blocked by unrelated generated Freemarker/calc symbols such as IFill, LinearFill, Accumulator, ComparatorChain, and IoTDBConfig.getModeMapSizeThreshold().
  • ConfigNode receiver records proxied through DataNode use receiver_node_id = -1 because the current proxy path does not expose the exact target ConfigNode id.

@Caideyipi
Caideyipi marked this pull request as ready for review June 11, 2026 10:46
@codecov

codecov Bot commented Jun 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 52.72145% with 443 lines in your changes missing coverage. Please review.
✅ Project coverage is 42.73%. Comparing base (9d91245) to head (72a0cc2).
⚠️ Report is 3 commits behind head on master.

Files with missing lines Patch % Lines
...eceiver/protocol/thrift/IoTDBDataNodeReceiver.java 10.92% 106 Missing ⚠️
...mmons/pipe/sink/client/IoTDBSyncClientManager.java 0.00% 60 Missing ⚠️
...ipe/receiver/protocol/IoTDBConfigNodeReceiver.java 0.00% 25 Missing ⚠️
...b/queryengine/plan/planner/LogicalPlanBuilder.java 0.00% 24 Missing ⚠️
...db/pipe/sink/protocol/writeback/WriteBackSink.java 62.00% 19 Missing ⚠️
...db/db/queryengine/plan/analyze/AnalyzeVisitor.java 0.00% 18 Missing ⚠️
...gent/task/subtask/sink/PipeSinkSubtaskManager.java 0.00% 14 Missing ⚠️
.../pipe/agent/task/subtask/sink/PipeSinkSubtask.java 13.33% 13 Missing ⚠️
...ueryengine/plan/planner/OperatorTreeGenerator.java 0.00% 13 Missing ⚠️
...e/iotdb/commons/pipe/agent/task/PipeTaskAgent.java 13.33% 13 Missing ⚠️
... and 31 more
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #17869      +/-   ##
============================================
+ Coverage     42.65%   42.73%   +0.07%     
- Complexity      414      442      +28     
============================================
  Files          5451     5459       +8     
  Lines        394061   395004     +943     
  Branches      51608    51700      +92     
============================================
+ Hits         168089   168791     +702     
- Misses       225972   226213     +241     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Caideyipi added 24 commits June 16, 2026 18:54
# Conflicts:
#	iotdb-core/datanode/src/test/java/org/apache/iotdb/db/pipe/receiver/protocol/thrift/IoTDBDataNodeReceiverTest.java
# Conflicts:
#	iotdb-core/relational-grammar/src/main/antlr4/org/apache/iotdb/db/relational/grammar/sql/RelationalSql.g4
…ow-receivers

# Conflicts:
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/sink/protocol/writeback/WriteBackSink.java
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/parser/ASTVisitor.java
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/security/TreeAccessCheckVisitor.java
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/statement/StatementVisitor.java
#	iotdb-core/datanode/src/test/java/org/apache/iotdb/db/pipe/sink/PipeSinkTest.java
# Conflicts:
#	iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/manager/pipe/receiver/protocol/IoTDBConfigNodeReceiver.java
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/receiver/protocol/thrift/IoTDBDataNodeReceiver.java
#	iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/pipe/receiver/IoTDBFileReceiver.java
…ow-receivers

# Conflicts:
#	iotdb-core/datanode/src/test/java/org/apache/iotdb/db/pipe/sink/PipeSinkTest.java
…ow-receivers

# Conflicts:
#	integration-test/src/test/java/org/apache/iotdb/pipe/it/single/IoTDBPipeReceiverIT.java
#	integration-test/src/test/java/org/apache/iotdb/relational/it/schema/IoTDBDatabaseIT.java
#	iotdb-core/datanode/src/test/java/org/apache/iotdb/db/pipe/receiver/protocol/airgap/IoTDBAirGapReceiverTest.java
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Comment on lines +62 to +70
public void testShowReceiversPipeIdsDisappearAfterDropPipe() throws Exception {
final String database = "root.show_receivers_lifecycle";
final String pipeName = "show_receivers_lifecycle_pipe";

createThriftPipe(database, pipeName);

assertShowReceivers("show receivers", BaseEnv.TREE_SQL_DIALECT, pipeName);
assertShowReceivers(
"select * from information_schema.receivers", BaseEnv.TABLE_SQL_DIALECT, pipeName);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can a tree pipe be shown in the table model?

Comment on lines +1183 to +1203
private TPipeTransferResp recordConfigNodeReceiverRuntimeIfSuccess(
final Pair<TPipeTransferResp, Integer> respWithReceiverNodeId, final TPipeTransferReq req) {
final TPipeTransferResp resp = respWithReceiverNodeId.left;
if (!PipeRequestType.isValidatedRequestType(req.getType())) {
return resp;
}

final PipeRequestType requestType = PipeRequestType.valueOf(req.getType());
if (requestType == PipeRequestType.HANDSHAKE_CONFIGNODE_V1
|| requestType == PipeRequestType.HANDSHAKE_CONFIGNODE_V2) {
if (isSuccess(resp)) {
recordConfigNodeHandshake(req, requestType, respWithReceiverNodeId.right);
}
} else {
if (isSuccess(resp)) {
PipeReceiverRuntimeRegistry.getInstance()
.markTransfer(configPipeReceiverRuntimeSessionKey.get(), System.currentTimeMillis());
}
}
return resp;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The return value is fixed. Is it necessary?

Comment on lines +1230 to +1231
getSenderHost(),
parseSenderPort(getSenderPort()),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are both needed.

Comment on lines +1243 to +1246
final Map<String, String> params = new HashMap<>();
if (req.getBody() == null) {
return params;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move the judgment to the very beginning and return Collections.emptyMap if the map will not be written later.

…ow-receivers

# Conflicts:
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/agent/task/subtask/sink/PipeSinkSubtaskManager.java
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/protocol/client/ConfigNodeClient.java
#	iotdb-core/datanode/src/main/java/org/apache/iotdb/db/protocol/client/ConfigNodeInfo.java
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants