#################################################################
# initial checking command
#################################################################
# isi status
# isi services -a | grep -i esrs
# isi_for_array -s ps auwxx | grep -i esrs | grep -v grep
# isi network pools list -v
# isi license list
# isi esrs view
Note the Primary Gateway IP from the output of above command.
# isi_for_array -s 'nc -z <Primary_Gateway_IP> 9443'
# isi_for_array -s 'nc -z <Primary_Gateway_IP> 443'
# isi_for_array -s 'nc -z <Primary_Gateway_IP> 8118'
#################################################################
# Restart ESRS service
#################################################################
Please run the below command to restart isi_esrs_d and isi_rsapi_d:
# isi services -a isi_esrs_d disable
# isi services -a isi_rsapi_d disable
After 30 sec run the below:
# isi services -a isi_esrs_d enable
# isi services -a isi_rsapi_d enable
and check #isi esrs view if gateway connectivity is showing as connected, if not then try the below:
# isi esrs modify --enabled=true --primary-esrs-gateway=x.x.x.x --gateway-access-pools=x.x --username < customer's support site account>
and then check gateway Connectivity status and share the output of the above.
*** isi esrs modify --enabled=true --username=Global_team@email.com --password=*********
----------------------------------------------------------------------------------
Sample Case: SR# 188834247
From the issue description, I understand that the following issue has been detected on cluster A***ISI01-4 .
Critical Events FAIL
FAIL: Critical event 1049042 for node All: 3 attempts to contact the configured ESRS gateway(s) failed
FAIL: Critical event 1054478 for node 4: Unhealthy boot disks (data drive containing var1), mirror is degraded or missing components.
INFO: For more information refer to KB article 000050913 found at https://www.dell.com/support/kbdoc/000050913.
As per the latest logs
Before:
A***ISI01-4-4]$ cat gmirror_status
Name Status Components
mirror/root1 COMPLETE ada0p5 (ACTIVE)
ada1p5 (ACTIVE)
mirror/hw COMPLETE ada1p14 (ACTIVE)
ada0p14 (ACTIVE)
mirror/keystore COMPLETE ada1p13 (ACTIVE)
ada0p13 (ACTIVE)
mirror/journal-backup COMPLETE ada1p12 (ACTIVE)
ada0p12 (ACTIVE)
mirror/var-crash COMPLETE ada1p10 (ACTIVE)
ada0p10 (ACTIVE)
mirror/mfg COMPLETE ada1p8 (ACTIVE)
ada0p8 (ACTIVE)
mirror/var1 DEGRADED da34p3 (ACTIVE)
mirror/var0 COMPLETE da35p3 (ACTIVE)
da36p3 (ACTIVE)
mirror/root0 COMPLETE ada1p4 (ACTIVE)
ada0p4 (ACTIVE)
Kindly login to Node 4 and at Node 4' prompt run the following command
isi_var_migrate --rebalance
post that check the status of boot partitions
gmirror status
After:
A***ISI01-4-4# isi_var_migrate --rebalance
No drives added to var0, 2 copies already exist
Adding ['da32'] to var1
Deleting da33p3
A***ISI01-4# gmirror status
Name Status Components
mirror/root1 COMPLETE ada0p5 (ACTIVE)
ada1p5 (ACTIVE)
mirror/hw COMPLETE ada1p14 (ACTIVE)
ada0p14 (ACTIVE)
mirror/keystore COMPLETE ada1p13 (ACTIVE)
ada0p13 (ACTIVE)
mirror/journal-backup COMPLETE ada1p12 (ACTIVE)
ada0p12 (ACTIVE)
mirror/var-crash COMPLETE ada1p10 (ACTIVE)
ada0p10 (ACTIVE)
mirror/mfg COMPLETE ada1p8 (ACTIVE)
ada0p8 (ACTIVE)
mirror/var1 COMPLETE da34p3 (ACTIVE)
da32p3 (ACTIVE)
mirror/var0 COMPLETE da35p3 (ACTIVE)
da36p3 (ACTIVE)
mirror/root0 COMPLETE ada1p4 (ACTIVE)
ada0p4 (ACTIVE)