@@ -1435,13 +1435,15 @@ Cartridge displays cluster and instances issues in WebUI:
1435
1435
* Failover:
1436
1436
1437
1437
* **warning **: "Can't obtain failover coordinator (...)";
1438
- * **warning **: "There is no active failover coordinator";
1438
+ * **warning **: "There is no active failover coordinator"
1439
+ -- all of the cooridinator instances are probably dead or unavailable;
1439
1440
* **warning **: "Failover is stuck on ...: Error fetching appointments (...)";
1440
1441
* **warning **: "Failover is stuck on ...: Failover fiber is dead" -- this is likely a bug;
1441
1442
1442
1443
* Switchover:
1443
1444
1444
- * **warning **: "Consistency on ... isn't reached yet";
1445
+ * **warning **: "Consistency on ... isn't reached yet" -- wait for the switchover to finish
1446
+ or try to restart an instance;
1445
1447
1446
1448
* Clock:
1447
1449
@@ -1451,15 +1453,18 @@ Cartridge displays cluster and instances issues in WebUI:
1451
1453
1452
1454
* **critical **: "Running out of memory on ..." -- when all 3 metrics
1453
1455
``items_used_ratio ``, ``arena_used_ratio ``, ``quota_used_ratio `` from
1454
- ``box.slab.info() `` exceed ``limits.fragmentation_threshold_critical ``;
1456
+ ``box.slab.info() `` exceed ``limits.fragmentation_threshold_critical ``.
1457
+ You need to add more memory to the instance;
1455
1458
1456
1459
* **warning **: "Memory is highly fragmented on ..." - when
1457
1460
``items_used_ratio > limits.fragmentation_threshold_warning `` and
1458
- both ``arena_used_ratio ``, ``quota_used_ratio `` exceed critical limit;
1461
+ both ``arena_used_ratio ``, ``quota_used_ratio `` exceed critical limit.
1462
+ You can restart the instance to defragment memory or change ``box.cfg.min_tuple_size `` or
1463
+ ``box.cfg.slab_alloc_factor `` to reduce fragmentation;
1459
1464
1460
1465
* Configuration:
1461
1466
1462
- * **warning **: "Configuration checksum mismatch on ...";
1467
+ * **warning **: "Configuration checksum mismatch on ..." -- config is probably corrupted somehow ;
1463
1468
* **warning **: "Configuration is prepared and locked on ...";
1464
1469
* **warning **: "Advertise URI (...) differs from clusterwide config (...)";
1465
1470
* **warning **: "Configuring roles is stuck on ... and hangs for ... so far";
@@ -1483,14 +1488,19 @@ Cartridge displays cluster and instances issues in WebUI:
1483
1488
1484
1489
* various vshard alerts (see vshard docs for details);
1485
1490
1486
- * **warning **: warning: "Group "..." wasn't bootstrapped: ...";
1491
+ * **warning **: "Group "..." wasn't bootstrapped: ...".
1492
+ You need to bootstrap the group again;
1487
1493
1488
- * **warning **: Vshard storages in replicaset %s marked as "all writable".
1494
+ * **warning **: "Vshard storages in replicaset ... marked as "all writable".
1495
+ You can fix it by setting ``all_rw = false `` in the replicaset configuration;
1489
1496
1490
1497
* Alien members:
1491
1498
1492
1499
* **warning **: "Instance ... with alien uuid is in the membership" --
1493
- when two separate clusters share the same cluster cookie;
1500
+ when two separate clusters share the same cluster cookie or
1501
+ when two clustes have the same first 32 symbols of cluster cookie.
1502
+ You need to change the cluster cookie on one of the clusters or set
1503
+ ``set_cookie_hash_membership `` in ``cartridge.cfg `` to ``true ``;
1494
1504
1495
1505
.. image :: images/cartridge-issues-alien-uuid.png
1496
1506
:align: left
@@ -1501,11 +1511,13 @@ Cartridge displays cluster and instances issues in WebUI:
1501
1511
* Expelled instances:
1502
1512
1503
1513
* **warning **: "Replicaset ... has expelled instance ... in box.space._cluster" -
1504
- when instance was expelled from replicaset, but still remains in box.space._cluster;
1514
+ when instance was expelled from replicaset, but still remains in box.space._cluster.
1515
+ You need to remove it manually;
1505
1516
1506
1517
* Deprecated space format:
1507
1518
1508
- * **warning **: "Instance ... has spaces with deprecated format: space1, ..."
1519
+ * **warning **: "Instance ... has spaces with deprecated format: space1, ...". You need to
1520
+ convert spaces to the new format;
1509
1521
1510
1522
* Raft issues:
1511
1523
0 commit comments