[fix](regression) Stabilize delete bitmap cache metrics test

[shuke987]

What problem does this PR solve?

Issue Number: N/A

Related PR: #64515

Problem Summary:

test_delete_bitmap_metrics verifies each replica's durable delete bitmap and lazy aggregate-cache snapshot. In multi-replica P0 environments, the case warmed cache entries before iteration and fetched the large global /brpc_metrics payload inside the per-replica loop. On build 202230, the third replica was inspected about 32 seconds after warm-up; its durable bitmap was still correct (7/7) but the capacity-bounded LRU snapshot was empty.

This change sorts tablet rows by ReplicaId to match use_fix_replica ordering, pins and queries each replica immediately before checking that same replica's cache, restores the session variable in finally, and fetches the global metrics payload once after all replica checks.

The patched single case passed 3 consecutive lock-serialized runs on the authorized agent5 test cluster.

Release note

None

Check List (For Author)

• Test

  • Regression test
  • Unit Test
  • Manual test (add detailed scripts or steps below)
  • No need to test or manual test. Explain why:
    • This is a refactor/code format and no logic has been changed.
    • Previous test can cover this change.
    • No code files have been changed.
    • Other reason

• Behavior changed:

  • No.
  • Yes.

• Does this need documentation?

  • No.
  • Yes.

Check List (For Reviewer who merge this PR)

• Confirm the release note
• Confirm test cases
• Confirm document
• Add branch pick label

[shuke987]

/review

[shuke987]

run buildall

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?

[github-actions]

Automated review completed after two rounds. I found one blocking issue: cloud multi-replica mode can still inspect a different BE's local aggregate-cache than the BE that served the warmup scan. I left the inline comment at the warmup query.

Checkpoint conclusions:

• Goal/test coverage: this one-file regression change stabilizes the local and normal-cloud paths, but it does not prove the intended per-BE aggregate-cache behavior when cloud multi-replica placement is enabled.
• Scope/focus: the change is small and focused; no additional user focus was provided.
• Concurrency/lifecycle/config/compatibility/persistence/data writes/FE-BE variables: no product-code or protocol changes; the relevant config-sensitive path is cloud multi-replica selection.
• Parallel paths: local/shared-nothing, normal cloud, and cloud multi-replica paths were checked; cloud multi-replica is the remaining gap.
• Regression-test standards/results: the deterministic result check remains ordered; the added warmup query is not an output assertion; no .out file was changed. No tests/builds were run in this review environment per the prompt.
• Observability/performance: moving /brpc_metrics polling out of the replica loop preserves the same fixed-backend metric coverage and avoids repeated large metrics scrapes.

Round 2 convergence reviewers returned NO_NEW_VALUABLE_FINDINGS for the current ledger/comment set. Requesting changes for the inline issue.

[github-actions]

This still looks flaky when cloud multi-replica placement is enabled. The test records tablet.BackendId from SHOW TABLETS, then warms with use_fix_replica=${ri} and probes /count_agg_cache on that stored backend. In cloud mode, though, CloudTablet has one logical CloudReplica, and CloudReplica.getBackendIdImpl() chooses a physical BE with rand.nextInt(Config.cloud_replica_num) on each backend-resolution call when enable_cloud_multi_replica is true. The SHOW TABLETS call and the later scan can therefore pick different physical BEs, while /count_agg_cache only snapshots the target BE's local agg cache. Please either pin/disable cloud multi-replica for this per-BE cache assertion, or query the actual BE selected by the warmup scan before checking its cache.

[shuke987]

/review

[shuke987]

run buildall

[github-actions]

Automated review completed after two rounds. I found one blocking issue: tagging this suite nonConcurrent makes it a SINGLE suite, while the checked-in cloud P0 runner disables SINGLE execution, so the cloud-specific fix loses its cloud CI path. I left one inline comment.

Checkpoint conclusions:

• Goal and test proof: the immediate per-replica warm-up and temporary cloud routing pin address the LRU/backend-mismatch mechanisms when the suite executes, but removing the suite from cloud P0 means the stated cloud behavior is not continuously proved.
• Scope and clarity: the implementation is focused in one regression suite, but the group-tag change has a non-local scheduling effect that defeats part of the goal.
• Concurrency, lifecycle, and configuration: the existing mutable FE config is restored by setFeConfigTemporary; nested finally blocks restore use_fix_replica and the BE polling interval. No lock, static-lifetime, or new-config issue was found. Serializing the global mutation is appropriate, but the cloud runner must actually execute that serial phase.
• Compatibility and parallel paths: there is no production API, storage format, persistence, transaction, or FE-BE variable change. Shared-nothing, normal cloud, and cloud multi-replica paths were checked; cloud CI reachability is the remaining issue.
• Tests and results: the ordered qt_sql check, all local/MS/cache assertions, retry behavior, and unchanged expected output are preserved. Per the review contract, no build or runtime test was run in this review environment; the current head does not demonstrate a cloud runtime rerun.
• Observability and performance: existing logs remain sufficient, and moving /brpc_metrics polling outside the replica loop preserves its assertions while avoiding redundant large scrapes.
• User focus: no additional user-provided focus was supplied.

Round 2 normal and risk-focused convergence reviewers returned NO_NEW_VALUABLE_FINDINGS for this exact ledger/comment set. Requesting changes for MAIN-001.

[github-actions]

Adding nonConcurrent makes this suite a SINGLE suite, but regression-test/pipeline/cloud_p0/run.sh invokes the framework with -runNonConcurrent false, so RegressionTest.runScripts returns before this suite can run. The standalone nonConcurrent pipeline is shared-nothing, and vault P0 only scans vault_p0, so the new isCloudMode()/multi-replica fix loses its checked-in cloud coverage. Please keep the global config mutation serialized while retaining a cloud execution path, for example with a cloud SINGLE phase/job or a deterministic binding that does not require reclassifying this suite.