[server] Add Cluster Health API implementation#3400
Conversation
swuferhong
force-pushed
the
fluss-server-recovery
branch
from
fe0c5c1 to
042fc7a
Compare
loserwang1024
left a comment
loserwang1024
left a comment
There was a problem hiding this comment.
I have left some comment.
| // re-election via onReplicaBecomeOffline. | ||
| List<NotifyLeaderAndIsrResultForBucket> failedResults = | ||
| new ArrayList<>(); | ||
| ApiError sendError = |
There was a problem hiding this comment.
Currently, after this modification, even a network timeout exception, the CoordinatorEventProcessor#processNotifyLeaderAndIsrResponseReceivedEvent will mark the server as offlineReplicas
won't is any problem?
There was a problem hiding this comment.
Good catch. I have reverted it — the original "just ignore" approach was intentional (the coordinator detects actual server death via heartbeat timeout). The only thing I kept is clearing the pendingLeaderActivationBuckets on send failure, so the health API doesn't report stale RED for buckets that are simply stuck in the pending set. Actual failover is still handled by the existing heartbeat/offline mechanism.
| echo "advertised.listeners: ${ADVERTISED_LISTENERS}" >> $FLUSS_HOME/conf/server.yaml && \ | ||
|
|
||
| bin/coordinator-server.sh start-foreground | ||
| exec bin/coordinator-server.sh start-foreground |
There was a problem hiding this comment.
why need to change it?
There was a problem hiding this comment.
Without exec, the Fluss process is not PID 1 in the pod, so graceful shutdown signals won't reach it. I fixed this as well.
swuferhong
force-pushed
the
fluss-server-recovery
branch
from
64f2544 to
c23ee95
Compare
|
@loserwang1024 comments addressed. PLTA. |
2 participants
Purpose
Linked issue: close #3399
GetClusterHealthRPC to Coordinator that computes cluster health from in-memory stateCoordinatorContext(marked inactive onNotifyLeaderAndIsrsend,marked active on successful response when responding server is still the leader)
CoordinatorRequestBatchby synthesizing error responses to clearpending inactive state
Admin.getClusterHealth()withClusterHealth/ClusterHealthStatustypesClusterHealthReadinessCheckCLI tool in fluss-dist (exit 0=GREEN, 1=not ready, 2=API unsupported)readiness-check.shtwo-step readiness probe script (TCP + Cluster Health API)with first-boot detection and grace period for API-unsupported (mixed-version rolling upgrade)
readiness-check.shin Helm chartBrief change log
Tests
API and Format
Documentation