Skip to content

Pipe: add ConfigNode memory control for log reducer#18069

Open
Caideyipi wants to merge 1 commit into
apache:masterfrom
Caideyipi:codex/confignode-pipe-log-reducer-memory
Open

Pipe: add ConfigNode memory control for log reducer#18069
Caideyipi wants to merge 1 commit into
apache:masterfrom
Caideyipi:codex/confignode-pipe-log-reducer-memory

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Summary

  • Add ConfigNodeMemoryConfig and confignode_memory_proportion for ConfigNode Pipe/Free memory split.
  • Wire PipePeriodicalLogReducer to DataNode and ConfigNode memory resize hooks.
  • Keep ConfigNode Pipe memory exposed only as logReducerMemory, so no other CN module is attached to the new memory framework.

Tests

  • PASS: mvn spotless:apply -pl iotdb-core/node-commons,iotdb-core/confignode,iotdb-core/datanode
  • PASS: mvn test -pl iotdb-core/node-commons,iotdb-core/confignode "-Dtest=PipePeriodicalLogReducerTest,ConfigNodeMemoryConfigTest" -DfailIfNoTests=false
  • PASS: mvn compile -pl iotdb-core/node-commons,iotdb-core/confignode
  • NOTE: mvn compile -pl iotdb-core/node-commons,iotdb-core/datanode reaches javac but fails on existing calc MemoryReservationManager#setHighestPriority mismatch unrelated to this Pipe change.

Comment on lines +41 to +43
String memoryAllocateProportion = properties.getProperty("confignode_memory_proportion", null);
if (memoryAllocateProportion == null) {
memoryAllocateProportion = properties.getProperty("config_node_memory_proportion", null);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just keep one

Comment on lines +44 to +48
if (memoryAllocateProportion != null) {
LOGGER.warn(
"The parameter config_node_memory_proportion is deprecated, "
+ "please use confignode_memory_proportion instead.");
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i18n

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants