RAC环境集群组件gpnp未启动成功问题分析

近期，某用户的RAC环境出现重启主机后未启动成功的问题。

1.排查集群启动日志

可以发现GPNP进程启动异常：

2022-02-23 04:01:46.163 [EVMD(5243)]CRS-8500: Oracle Clusterware EVMD process is starting with operating system pro
cess ID 5243
2022-02-23 04:01:47.182 [GPNPD(5276)]CRS-8500: Oracle Clusterware GPNPD process is starting with operating system p
rocess ID 5276
2022-02-23 04:01:48.154 [GPNPD(5276)]CRS-8501: Oracle Clusterware GPNPD process with operating system process ID 5276 is ending with return value 1   ====>>>返回值异常
2022-02-23 04:01:48.198 [GPNPD(5342)]CRS-8500: Oracle Clusterware GPNPD process is starting with operating system p
rocess ID 5342
2022-02-23 04:01:48.200 [GIPCD(5333)]CRS-8500: Oracle Clusterware GIPCD process is starting with operating system p
rocess ID 5333

2.检查异常的GPNP进程TRACE日志

报错没有检测到网络接口信息，所以GPNP进程终止。

2022-02-23 04:01:34.660 : default:923271232: GPNPD tracelevel 1
2022-02-23 04:01:34.661 :    GPNP:923271232: clsgpnpd_lOpenEP: [at clsgpnpd.c:1943] Listening on "ipc://GPNPD_zgj-bdc-zzs-db02"
2022-02-23 04:01:34.662 :  CLSDMT:753927936: PID for the Process [3083], connkey GPNPD
2022-02-23 04:01:35.632 :    GPNP:923271232: clsgpnpd_validateProfile: [at clsgpnpdcmn.c:1013] GPnPD taken cluster guid '2ff42db654746f0cbf89a0a7ff0038e8'
2022-02-23 04:01:35.632 :    GPNP:923271232: clsgpnpd_validateProfile: [at clsgpnpdcmn.c:1040] GPnPD taken cluster name 'zgj-bdc-cluster'
2022-02-23 04:01:35.632 :    GPNP:923271232: clsgpnpd_openLocalProfile: [at clsgpnpd.c:2280] Got local profile from file cache provider (LCP-FS).
2022-02-23 04:01:35.633 :    GPNP:923271232: clsgpnpd_openLocalProfile: [at clsgpnpd.c:2337] Got local profile fromLR cache provider (LCP-OLR).
2022-02-23 04:01:35.634 :    GPNP:923271232: clsgpnpd_lCheckIpTypes: [at clsgpnpd.c:1644] Profile Networks Definitions - 2 total
2022-02-23 04:01:35.634 :    GPNP:923271232: clsgpnpd_lFilterIpTypes: [at clsgpnpd.c:1551]   - eth0/10.100.18.0 public (ip=,mask=,mac=,typ=1) 
2022-02-23 04:01:35.634 :    GPNP:923271232: clsgpnpd_lFilterIpTypes: [at clsgpnpd.c:1551]   - eth1/10.100.19.0 cluster_interconnect,asm (ip=,mask=,mac=,typ=1) 
2022-02-23 04:01:35.634 :    GPNP:923271232: clsgpnpd_lFilterIpTypes: [at clsgpnpd.c:1568]   of 2 net interfaces, 1 publics (1 ipv4, 0 ipv6), 1 privates (1 ipv4, 0 ipv6).
2022-02-23 04:01:35.634 :    GPNP:923271232: clsgpnpd_lCheckIpTypes: [at clsgpnpd.c:1677] Result: (1) CLSGPNP_ERR. (:GPNPD00121:) clsinet_ProfileGetNetData() returned no network interface definitions. Current node network interfaces do not map to GPnP Profile networks.  Check network hardware setup. ===>>>>报错没有检测到网络接口信息
2022-02-23 04:01:35.634 :    GPNP:923271232: clsgpnpd_term: [at clsgpnpd.c:1131] STOP GPnPD terminating. Closing connections...
2022-02-23 04:01:35.636 : default:923271232: clsgpnpd_term STOP terminating.

3.网络的排查

问题发生后，查看网卡状态是正常的。后经分析，原因为虚拟化整体发生重启；主机重启后虚拟化的虚拟网络有问题还未启动。后续网络正常后，KILL 异常的CRS相关信息，重新启动CRS软件，恢复正常。