Skip to content

Improve pipe runtime memory allocation#18051

Open
Caideyipi wants to merge 4 commits into
masterfrom
pipe-dynamic-memory-allocation
Open

Improve pipe runtime memory allocation#18051
Caideyipi wants to merge 4 commits into
masterfrom
pipe-dynamic-memory-allocation

Fix TsFile parser OOM retry test setup

50de9b2
Select commit
Loading
Failed to load commit list.
SonarQubeCloud / SonarCloud Code Analysis failed Jul 1, 2026 in 2m 49s

Quality Gate failed

Failed conditions
11.3% Duplication on New Code (required ≤ 5%)

See analysis details on SonarQube Cloud

Annotations

Check warning on line 512 in iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/sink/protocol/iotconsensusv2/IoTConsensusV2SyncSink.java

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Remove this unnecessary cast to "long". 

See more on https://sonarcloud.io/project/issues?id=apache_iotdb&issues=AZ8Rr3gcfwQ4mSZzoAL2&open=AZ8Rr3gcfwQ4mSZzoAL2&pullRequest=18051

Check warning on line 468 in iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/sink/protocol/legacy/IoTDBLegacyPipeSink.java

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Remove this unnecessary cast to "long". 

See more on https://sonarcloud.io/project/issues?id=apache_iotdb&issues=AZ8Rr3iEfwQ4mSZzoAL9&open=AZ8Rr3iEfwQ4mSZzoAL9&pullRequest=18051

Check warning on line 449 in iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tsfile/parser/scan/TsFileInsertionEventScanParser.java

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

A "Brain Method" was detected. Refactor it to reduce at least one of the following metrics: LOC from 134 to 64, Complexity from 34 to 14, Nesting Level from 4 to 2, Number of Variables from 8 to 6. 

See more on https://sonarcloud.io/project/issues?id=apache_iotdb&issues=AZ8Rr3kEfwQ4mSZzoAMB&open=AZ8Rr3kEfwQ4mSZzoAMB&pullRequest=18051

Check warning on line 529 in iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tsfile/PipeTsFileInsertionEvent.java

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

A "Brain Method" was detected. Refactor it to reduce at least one of the following metrics: LOC from 100 to 64, Complexity from 21 to 14, Nesting Level from 5 to 2, Number of Variables from 13 to 6. 

See more on https://sonarcloud.io/project/issues?id=apache_iotdb&issues=AZ8bkCfStSRAbF5ZBDRY&open=AZ8bkCfStSRAbF5ZBDRY&pullRequest=18051

Check warning on line 134 in iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/sink/protocol/thrift/async/handler/PipeTransferTsFileHandler.java

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Remove this unnecessary cast to "long". 

See more on https://sonarcloud.io/project/issues?id=apache_iotdb&issues=AZ8Rr3g9fwQ4mSZzoAL3&open=AZ8Rr3g9fwQ4mSZzoAL3&pullRequest=18051

Check warning on line 750 in iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tsfile/PipeTsFileInsertionEvent.java

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Replace this instanceof check and cast with 'instanceof PipeRawTabletInsertionEvent piperawtabletinsertionevent' 

See more on https://sonarcloud.io/project/issues?id=apache_iotdb&issues=AZ8bkCfStSRAbF5ZBDRW&open=AZ8bkCfStSRAbF5ZBDRW&pullRequest=18051

Check warning on line 730 in iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tsfile/PipeTsFileInsertionEvent.java

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Extract this nested try block into a separate method. 

See more on https://sonarcloud.io/project/issues?id=apache_iotdb&issues=AZ8bkCfStSRAbF5ZBDRT&open=AZ8bkCfStSRAbF5ZBDRT&pullRequest=18051

Check warning on line 989 in iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/agent/task/PipeDataNodeTaskAgent.java

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Remove this unused private "calculateSendTsFileReadBufferMemory" method. 

See more on https://sonarcloud.io/project/issues?id=apache_iotdb&issues=AZ8Rr3jGfwQ4mSZzoAMA&open=AZ8Rr3jGfwQ4mSZzoAMA&pullRequest=18051

Check failure on line 149 in iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/processor/downsampling/DownSamplingProcessor.java

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Refactor this method to reduce its Cognitive Complexity from 19 to the 15 allowed. 

See more on https://sonarcloud.io/project/issues?id=apache_iotdb&issues=AZ8bkCgLtSRAbF5ZBDRa&open=AZ8bkCgLtSRAbF5ZBDRa&pullRequest=18051

Check warning on line 119 in iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/sink/protocol/iotconsensusv2/handler/IoTConsensusV2TsFileInsertionEventHandler.java

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Remove this unnecessary cast to "long". 

See more on https://sonarcloud.io/project/issues?id=apache_iotdb&issues=AZ8Rr3b1fwQ4mSZzoAL1&open=AZ8Rr3b1fwQ4mSZzoAL1&pullRequest=18051

Check warning on line 680 in iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/sink/protocol/thrift/sync/IoTDBDataRegionSyncSink.java

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Remove this unnecessary cast to "long". 

See more on https://sonarcloud.io/project/issues?id=apache_iotdb&issues=AZ8Rr3hUfwQ4mSZzoAL7&open=AZ8Rr3hUfwQ4mSZzoAL7&pullRequest=18051

Check warning on line 750 in iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tsfile/PipeTsFileInsertionEvent.java

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Replace this instanceof check and cast with 'instanceof PipeRawTabletInsertionEvent piperawtabletinsertionevent' 

See more on https://sonarcloud.io/project/issues?id=apache_iotdb&issues=AZ8bkCfStSRAbF5ZBDRV&open=AZ8bkCfStSRAbF5ZBDRV&pullRequest=18051

Check warning on line 145 in iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/sink/protocol/thrift/async/handler/PipeTransferTsFileHandler.java

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

A "Brain Method" was detected. Refactor it to reduce at least one of the following metrics: LOC from 90 to 64, Complexity from 15 to 14, Nesting Level from 3 to 2, Number of Variables from 14 to 6. 

See more on https://sonarcloud.io/project/issues?id=apache_iotdb&issues=AZ8Rr3g9fwQ4mSZzoAL4&open=AZ8Rr3g9fwQ4mSZzoAL4&pullRequest=18051

Check warning on line 615 in iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/sink/protocol/thrift/sync/IoTDBDataRegionSyncSink.java

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Reduce the total number of break and continue statements in this loop to use at most one. 

See more on https://sonarcloud.io/project/issues?id=apache_iotdb&issues=AZ8Rr3hUfwQ4mSZzoAL5&open=AZ8Rr3hUfwQ4mSZzoAL5&pullRequest=18051

Check warning on line 750 in iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tsfile/PipeTsFileInsertionEvent.java

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Replace this instanceof check and cast with 'instanceof PipeRawTabletInsertionEvent piperawtabletinsertionevent' 

See more on https://sonarcloud.io/project/issues?id=apache_iotdb&issues=AZ8bkCfStSRAbF5ZBDRX&open=AZ8bkCfStSRAbF5ZBDRX&pullRequest=18051

Check failure on line 602 in iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/sink/protocol/thrift/sync/IoTDBDataRegionSyncSink.java

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Refactor this method to reduce its Cognitive Complexity from 16 to the 15 allowed. 

See more on https://sonarcloud.io/project/issues?id=apache_iotdb&issues=AZ8Rr3hUfwQ4mSZzoAL6&open=AZ8Rr3hUfwQ4mSZzoAL6&pullRequest=18051

Check warning on line 888 in iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/agent/task/PipeDataNodeTaskAgent.java

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Remove this unused private "calculateTsFileParserMemory" method. 

See more on https://sonarcloud.io/project/issues?id=apache_iotdb&issues=AZ8Rr3jGfwQ4mSZzoAL-&open=AZ8Rr3jGfwQ4mSZzoAL-&pullRequest=18051

Check warning on line 960 in iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/agent/task/PipeDataNodeTaskAgent.java

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Remove this unused private "calculateSinkBatchMemory" method. 

See more on https://sonarcloud.io/project/issues?id=apache_iotdb&issues=AZ8Rr3jGfwQ4mSZzoAL_&open=AZ8Rr3jGfwQ4mSZzoAL_&pullRequest=18051

Check warning on line 586 in iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tsfile/parser/scan/TsFileInsertionEventScanParser.java

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

A "Brain Method" was detected. Refactor it to reduce at least one of the following metrics: LOC from 123 to 64, Complexity from 25 to 14, Nesting Level from 4 to 2, Number of Variables from 12 to 6. 

See more on https://sonarcloud.io/project/issues?id=apache_iotdb&issues=AZ8Rr3kEfwQ4mSZzoAMC&open=AZ8Rr3kEfwQ4mSZzoAMC&pullRequest=18051

Check warning on line 737 in iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tsfile/PipeTsFileInsertionEvent.java

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Either log this exception and handle it, or rethrow it with some contextual information. 

See more on https://sonarcloud.io/project/issues?id=apache_iotdb&issues=AZ8bkCfStSRAbF5ZBDRU&open=AZ8bkCfStSRAbF5ZBDRU&pullRequest=18051

Check warning on line 542 in iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/sink/protocol/airgap/IoTDBDataRegionAirGapSink.java

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Remove this unnecessary cast to "long". 

See more on https://sonarcloud.io/project/issues?id=apache_iotdb&issues=AZ8Rr3hsfwQ4mSZzoAL8&open=AZ8Rr3hsfwQ4mSZzoAL8&pullRequest=18051

Check warning on line 428 in iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/processor/aggregate/AggregateProcessor.java

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

A "Brain Method" was detected. Refactor it to reduce at least one of the following metrics: LOC from 91 to 64, Complexity from 17 to 14, Nesting Level from 4 to 2, Number of Variables from 12 to 6. 

See more on https://sonarcloud.io/project/issues?id=apache_iotdb&issues=AZ8bkCf0tSRAbF5ZBDRZ&open=AZ8bkCf0tSRAbF5ZBDRZ&pullRequest=18051
View more details on SonarQubeCloud