Oracle11gRACCRS4535/ORA15077

oracle 11.2.0.4的rac集群，其中一个节点出现故障，集群无法启动，

使用crsctl check crs查看集群状况如下：

[[email&＃160;protected] client]$ crsctl check crs

CRS-4638: Oracle High Availability Services is online

CRS-4535: Cannot communicate with Cluster Ready Services

CRS-4529: Cluster Synchronization Services is online

ps -ef | grep d.bin #下面的查询中没有crsd.bin

查看tail -100 alertdb2.log

[client(193951)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/app/11.2.0/grid/log/db2/client/crsctl_oracle.log.

2016-11-22 16:37:37.539:

提示不能访问asm磁盘组中的OCR信息，详细信息查看client中的日志。

[[email&＃160;protected] client]$ tail -100 crsctl_oracle.log

2016-11-22 16:36:19.295: [ OCRASM][1798887200]ASM Error Stack : ORA-15077: could not locate ASM instance serving a required diskgroup

2016-11-22 16:36:19.302: [ OCRASM][1798887200]proprasmo: kgfoCheckMount returned [7]

2016-11-22 16:36:19.302: [ OCRASM][1798887200]proprasmo: The ASM instance is down

2016-11-22 16:36:19.506: [ OCRRAW][1798887200]proprioo: Failed to open [+OCR]. Returned proprasmo() with [26]. Marking location as UNAVAILABLE.

2016-11-22 16:36:19.506: [ OCRRAW][1798887200]proprioo: No OCR/OLR devices are usable

2016-11-22 16:36:19.506: [ OCRASM][1798887200]proprasmcl: asmhandle is NULL

2016-11-22 16:36:19.506: [ OCRRAW][1798887200]proprinit: Could not open raw device

2016-11-22 16:36:19.506: [ OCRASM][1798887200]proprasmcl: asmhandle is NULL

2016-11-22 16:36:19.506: [ default][1798887200]a_init:7!: Backend init unsuccessful : [26]

2016-11-22 16:36:19.736: [ OCRMSG][979273504]prom_waitconnect: CONN NOT ESTABLISHED (0,29,1,2)

2016-11-22 16:36:19.736: [ OCRMSG][979273504]GIPC error [29] msg [gipcretConnectionRefused]

2016-11-22 16:36:19.736: [ OCRMSG][979273504]prom_connect: error while waiting for connection complete [24]

[ CLWAL][979273504]clsw_Initialize: OLR initlevel [30000]

提示ASM实例关闭了，不能打开OCR。

解决：

[[email&＃160;protected] client]$ asmcmd

Connected to an idle instance.

ASMCMD> startup

ASM instance started

Total System Global Area 1703624704 bytes

Fixed Size 2253904 bytes

Variable Size 1667816368 bytes

ASM Cache 33554432 bytes

ASM diskgroups mounted

启动后，本地集群资源随即正常启动。