Created an API to fetch remote store metadata

[Sukriti0717]

Added Remote Store Metadata API to fetch segment and translog metadata

Description

This PR adds a new cluster-level API that enables users to retrieve remote store metadata, including segment and translog details, for specific indices and shards. This functionality enhances observability and debugging capabilities for indices backed by remote storage.

Key Changes

1. Transport Layer: TransportRemoteStoreMetadataAction:

• Executes on the Data node.
• Collects and parses segment and translog metadata files for each shard.
• Handles shard-level errors gracefully.

2. Core Action Classes

• RemoteStoreMetadataRequest: Supports input parameters: index name and optional shard ID(s).
• RemoteStoreMetadataResponse: Contains per-shard metadata along with success/failure info
• RemoteStoreMetadata: Holds the parsed segment and translog metadata per shard.
• RemoteStoreMetadataAction: Registers the action under cluster:admin/remote_store/metadata.
• RemoteStoreMetadataRequestBuilder: Provides fluent client-side request building.

3. REST API: RestRemoteStoreMetadataAction

• New endpoints:
  • GET /_remotestore/metadata/{index}
  • GET /_remotestore/metadata/{index}/{shard_id} - Supports optional query param: ?local=true
• Internally calls the transport action and returns structured JSON.

4. Client Integration

• Updates to ClusterAdminClient and AbstractClient:
  • remoteStoreMetadata(RemoteStoreMetadataRequest, ActionListener)
  • prepareRemoteStoreMetadata(String index, String shardId)

Sample Response

GET "localhost:9200/_remotestore/metadata/my-index-1?pretty"

{
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "indices" : {
    "my-index-1" : {
      "shards" : {
        "0" : [
          {
            "index" : "my-index-1",
            "shard" : 0,
            "segments" : {
              "files" : {
                "_0.cfe" : {
                  "original_name" : "_0.cfe",
                  "checksum" : "3262177161",
                  "length" : 517
                },
                "_0.si" : {
                  "original_name" : "_0.si",
                  "checksum" : "672251640",
                  "length" : 326
                },
                "_0.cfs" : {
                  "original_name" : "_0.cfs",
                  "checksum" : "3187604779",
                  "length" : 3933
                },
                "segments_3" : {
                  "original_name" : "segments_3",
                  "checksum" : "2519290512",
                  "length" : 230
                }
              },
              "replication_checkpoint" : {
                "segment_infos_version" : 9,
                "codec" : "Lucene101",
                "length" : 4776,
                "created_timestamp" : 28247749083916,
                "segments_gen" : 3,
                "primary_term" : 1
              }
            },
            "translog" : {
              "generation" : 3,
              "generation_to_primary_term" : {
                "3" : "1"
              },
              "min_translog_gen" : 3,
              "primary_term" : 1
            }
          }
        ]
      }
    }
  }

[github-actions]

❌ Gradle check result for 2e31b7f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for c3433c6: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 73c9694: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for f706bd0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for f368d9e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for c1e02e2: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for c013578: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 2a61841: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 432cc70: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for b3ed29d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 1476131: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for d92ef51: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for d92ef51: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for d92ef51: SUCCESS

[codecov]

Codecov Report

Attention: Patch coverage is 68.86792% with 66 lines in your changes missing coverage. Please review.

Project coverage is 72.60%. Comparing base (58c281f) to head (722d6dd).
Report is 62 commits behind head on main.

Files with missing lines	Patch %	Lines
...e/metadata/TransportRemoteStoreMetadataAction.java	50.49%	48 Missing and 2 partials ⚠️
...n/admin/cluster/RestRemoteStoreMetadataAction.java	33.33%	8 Missing ⚠️
...earch/transport/client/support/AbstractClient.java	0.00%	7 Missing ⚠️
...otestore/metadata/RemoteStoreMetadataResponse.java	97.05%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #18257      +/-   ##
============================================
+ Coverage     72.44%   72.60%   +0.16%     
- Complexity    67256    67475     +219     
============================================
  Files          5488     5495       +7     
  Lines        310994   311206     +212     
  Branches      45212    45231      +19     
============================================
+ Hits         225288   225940     +652     
+ Misses        67297    66892     -405     
+ Partials      18409    18374      -35     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

❌ Gradle check result for 4b55b62: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 722d6dd: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 722d6dd: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❕ Gradle check result for 722d6dd: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.