宕机重启oracle数据库,RAC一节点宕机重启问题

本帖最后由 zyo320 于 2012-11-16 12:44 编辑

RAC双节点&＃xff0c;数据库版本&＃xff1a;10.2.0.1 系统 Linux  2.6.18-128.7NFS #1 SMP

节点2&＃xff1a;

alert信息&＃xff1a;

Shutting down instance (abort)

License high water mark &＃61; 193

Instance terminated by USER, pid &＃61; 31102

Fri Nov 16 04:23:07 2012

Starting ORACLE instance (normal)

LICENSE_MAX_SESSION &＃61; 0

LICENSE_SESSIONS_WARNING &＃61; 0

Interface type 1 eth1 192.168.100.0 configured from OCR for use as a cluster interconnect

Interface type 1 eth0 10.0.0.0 configured from OCR for use as  a public interface

Picked latch-free SCN scheme 1

Autotune of undo retention is turned on.

LICENSE_MAX_USERS &＃61; 0

SYS auditing is disabled

ksdpec: called for event 13740 prior to event group initialization

Starting up ORACLE RDBMS Version: 10.2.0.1.0.

System parameters with non-default values:

processes                &＃61; 1000

sessions                 &＃61; 1105

__shared_pool_size       &＃61; 603979776

__large_pool_size        &＃61; 16777216

__java_pool_size         &＃61; 16777216

__streams_pool_size      &＃61; 0

spfile                   &＃61; &＃43;FOODSAFE_DISK/safedb/spfilesafedb.ora

sga_target               &＃61; 5016387584

control_files            &＃61; &＃43;FOODSAFE_DISK/safedb/controlfile/current.256.759414295

db_block_size            &＃61; 8192

__db_cache_size          &＃61; 4362076160

db_writer_processes      &＃61; 3

compatible               &＃61; 10.2.0.1.0

log_archive_dest_1       &＃61; location&＃61;/dbrac2_arch

log_archive_dest_2       &＃61;

log_archive_max_processes&＃61; 4

standby_archive_dest     &＃61;

log_checkpoint_interval  &＃61; 0

log_checkpoint_timeout   &＃61; 0

db_file_multiblock_read_count&＃61; 16

cluster_database         &＃61; TRUE

cluster_database_instances&＃61; 2

db_create_file_dest      &＃61; &＃43;FOODSAFE_DISK

thread                   &＃61; 2

fast_start_mttr_target   &＃61; 300

instance_number          &＃61; 2

undo_management          &＃61; AUTO

undo_tablespace          &＃61; UNDOTBS2

_kgl_large_heap_warning_threshold&＃61; 52428800

remote_login_passwordfile&＃61; EXCLUSIVE

db_domain                &＃61;

dispatchers              &＃61; (PROTOCOL&＃61;TCP) (SERVICE&＃61;SAFEDBXDB)

local_listener           &＃61; (ADDRESS&＃61;(PROTOCOL&＃61;TCP)(HOST&＃61;10.0.0.123)(PORT&＃61;1521))

remote_listener          &＃61; LISTENERS_SAFEDB

job_queue_processes      &＃61; 10

background_dump_dest     &＃61; /opt/ora10g/admin/SAFEDB/bdump

user_dump_dest           &＃61; /opt/ora10g/admin/SAFEDB/udump

core_dump_dest           &＃61; /opt/ora10g/admin/SAFEDB/cdump

audit_file_dest          &＃61; /opt/ora10g/admin/SAFEDB/adump

db_name                  &＃61; SAFEDB

open_cursors             &＃61; 300

pga_aggregate_target     &＃61; 1669332992

Cluster communication is configured to use the following interface(s) for this instance

192.168.100.101

Fri Nov 16 04:23:08 2012

cluster interconnect IPC version:Oracle UDP/IP

IPC Vendor 1 proto 2

PMON started with pid&＃61;2, OS id&＃61;31270

DIAG started with pid&＃61;3, OS id&＃61;31272

PSP0 started with pid&＃61;4, OS id&＃61;31274

LMON started with pid&＃61;5, OS id&＃61;31276

LMD0 started with pid&＃61;6, OS id&＃61;31278

LMS0 started with pid&＃61;7, OS id&＃61;31280

LMS1 started with pid&＃61;8, OS id&＃61;31284

LMS2 started with pid&＃61;9, OS id&＃61;31288

LMS3 started with pid&＃61;10, OS id&＃61;31292

MMAN started with pid&＃61;11, OS id&＃61;31296

DBW0 started with pid&＃61;12, OS id&＃61;31298

DBW1 started with pid&＃61;13, OS id&＃61;31300

DBW2 started with pid&＃61;14, OS id&＃61;31302

LGWR started with pid&＃61;15, OS id&＃61;31304

CKPT started with pid&＃61;16, OS id&＃61;31306

SMON started with pid&＃61;17, OS id&＃61;31308

RECO started with pid&＃61;18, OS id&＃61;31335

CJQ0 started with pid&＃61;19, OS id&＃61;31337

MMON started with pid&＃61;20, OS id&＃61;31339

Fri Nov 16 04:23:09 2012

starting up 1 dispatcher(s) for network address &＃39;(ADDRESS&＃61;(PARTIAL&＃61;YES)(PROTOCOL&＃61;TCP))&＃39;...

MMNL started with pid&＃61;21, OS id&＃61;31341

Fri Nov 16 04:23:09 2012

starting up 1 shared server(s) ...

Fri Nov 16 04:23:09 2012

lmon registered with NM - instance id 2 (internal mem no 1)

Fri Nov 16 04:23:10 2012

Reconfiguration started (old inc 0, new inc 48)

pseudo shared rm latch used

List of nodes:

0 1

Global Resource Directory frozen

* allocate domain 0, invalid &＃61; TRUE

Communication channels reestablished

* domain 0 valid &＃61; 0 according to instance 0

Fri Nov 16 04:23:11 2012

Master broadcasted resource hash value bitmaps

Non-local Process blocks cleaned out

Fri Nov 16 04:23:11 2012

LMS 1: 0 GCS shadows cancelled, 0 closed

Fri Nov 16 04:23:11 2012

LMS 2: 0 GCS shadows cancelled, 0 closed

Fri Nov 16 04:23:11 2012

LMS 3: 0 GCS shadows cancelled, 0 closed

Fri Nov 16 04:23:11 2012

LMS 0: 0 GCS shadows cancelled, 0 closed

Set master node info

Submitted all remote-enqueue requests

Dwn-cvts replayed, VALBLKs dubious

All grantable enqueues granted

Fri Nov 16 04:23:12 2012

LMS 2: 0 GCS shadows traversed, 0 replayed

Fri Nov 16 04:23:12 2012

LMS 0: 0 GCS shadows traversed, 0 replayed

Fri Nov 16 04:23:12 2012

LMS 3: 0 GCS shadows traversed, 0 replayed

Fri Nov 16 04:23:12 2012

LMS 1: 0 GCS shadows traversed, 0 replayed

Fri Nov 16 04:23:12 2012

Submitted all GCS remote-cache requests

Fix write in gcs resources

Reconfiguration complete

LCK0 started with pid&＃61;24, OS id&＃61;31422

Fri Nov 16 04:23:13 2012

ALTER DATABASE   MOUNT

Fri Nov 16 04:23:13 2012

Starting background process ASMB

ASMB started with pid&＃61;26, OS id&＃61;31427

Starting background process RBAL

RBAL started with pid&＃61;27, OS id&＃61;31441

Fri Nov 16 04:23:23 2012

SUCCESS: diskgroup FOODSAFE_DISK was mounted

Fri Nov 16 04:23:27 2012

Setting recovery target incarnation to 2

Fri Nov 16 04:23:27 2012

Successful mount of redo thread 2, with mount id 2685604006

Fri Nov 16 04:23:27 2012

Database mounted in Shared Mode (CLUSTER_DATABASE&＃61;TRUE)

Completed: ALTER DATABASE   MOUNT

Fri Nov 16 04:23:27 2012

ALTER DATABASE OPEN

Picked broadcast on commit scheme to generate SCNs

Fri Nov 16 04:23:28 2012

LGWR: STARTING ARCH PROCESSES

ARC0 started with pid&＃61;29, OS id&＃61;31802

ARC1 started with pid&＃61;30, OS id&＃61;31804

ARC2 started with pid&＃61;31, OS id&＃61;31806

Fri Nov 16 04:23:28 2012

ARC0: Archival started

ARC1: Archival started

ARC2: Archival started

ARC3: Archival started

LGWR: STARTING ARCH PROCESSES COMPLETE

ARC3 started with pid&＃61;32, OS id&＃61;31808

Fri Nov 16 04:23:28 2012

Thread 2 opened at log sequence 1956

Current log# 8 seq# 1956 mem# 0: &＃43;FOODSAFE_DISK/safedb/onlinelog/group_8.290.798134725

Successful open of redo thread 2

Fri Nov 16 04:23:28 2012

ARC2: STARTING ARCH PROCESSES

Fri Nov 16 04:23:28 2012

ARC1: Becoming the &＃39;no FAL&＃39; ARCH

ARC1: Becoming the &＃39;no SRL&＃39; ARCH

Fri Nov 16 04:23:28 2012

ARC0: Becoming the heartbeat ARCH

Fri Nov 16 04:23:28 2012

SMON: enabling cache recovery

Fri Nov 16 04:23:28 2012

ARC4: Archival started

ARC2: STARTING ARCH PROCESSES COMPLETE

ARC4 started with pid&＃61;33, OS id&＃61;31810

Fri Nov 16 04:23:29 2012

Successfully onlined Undo Tablespace 4.

Fri Nov 16 04:23:29 2012

SMON: enabling tx recovery

Fri Nov 16 04:23:29 2012

Database Characterset is ZHS16GBK

replication_dependency_tracking turned off (no async multimaster replication found)

Starting background process QMNC

QMNC started with pid&＃61;38, OS id&＃61;31871

Fri Nov 16 04:23:31 2012

Completed: ALTER DATABASE OPEN

Fri Nov 16 04:23:40 2012

ALTER SYSTEM SET service_names&＃61;&＃39;SAFEDB&＃39;,&＃39;foodsafe&＃39; SCOPE&＃61;MEMORY SID&＃61;&＃39;SAFEDB2&＃39;;

Fri Nov 16 04:29:28 2012

Shutting down archive processes

Fri Nov 16 04:29:33 2012

ARCH shutting down

ARC4: Archival stopped

节点1&＃xff1a;alert信息

Fri Nov 16 04:23:08 2012

Reconfiguration started (old inc 44, new inc 46)

List of nodes:

0

Global Resource Directory frozen

* dead instance detected - domain 0 invalid &＃61; TRUE

Communication channels reestablished

Master broadcasted resource hash value bitmaps

Non-local Process blocks cleaned out

Fri Nov 16 04:23:08 2012

LMS 2: 0 GCS shadows cancelled, 0 closed

Fri Nov 16 04:23:08 2012

LMS 1: 0 GCS shadows cancelled, 0 closed

Fri Nov 16 04:23:08 2012

LMS 3: 0 GCS shadows cancelled, 0 closed

Fri Nov 16 04:23:08 2012

LMS 0: 0 GCS shadows cancelled, 0 closed

Set master node info

Submitted all remote-enqueue requests

Dwn-cvts replayed, VALBLKs dubious

All grantable enqueues granted

Post SMON to start 1st pass IR

Fri Nov 16 04:23:08 2012

LMS 3: 90512 GCS shadows traversed, 0 replayed

Fri Nov 16 04:23:08 2012

LMS 2: 90141 GCS shadows traversed, 0 replayed

Fri Nov 16 04:23:08 2012

LMS 1: 90903 GCS shadows traversed, 0 replayed

Fri Nov 16 04:23:08 2012

LMS 0: 93644 GCS shadows traversed, 0 replayed

Fri Nov 16 04:23:08 2012

Submitted all GCS remote-cache requests

Post SMON to start 1st pass IR

Fix write in gcs resources

Fri Nov 16 04:23:08 2012

Instance recovery: looking for dead threads

Fri Nov 16 04:23:08 2012

Beginning instance recovery of 1 threads

Reconfiguration complete

Fri Nov 16 04:23:10 2012

Reconfiguration started (old inc 46, new inc 48)

List of nodes:

0 1

Global Resource Directory frozen

Communication channels reestablished

Master broadcasted resource hash value bitmaps

Non-local Process blocks cleaned out

Fri Nov 16 04:23:11 2012

LMS 1: 0 GCS shadows cancelled, 0 closed

Fri Nov 16 04:23:11 2012

LMS 0: 0 GCS shadows cancelled, 0 closed

Fri Nov 16 04:23:11 2012

LMS 3: 0 GCS shadows cancelled, 0 closed

Fri Nov 16 04:23:11 2012

LMS 2: 0 GCS shadows cancelled, 0 closed

Set master node info

Submitted all remote-enqueue requests

Dwn-cvts replayed, VALBLKs dubious

All grantable enqueues granted

Fri Nov 16 04:23:12 2012

LMS 0: 8788 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 3: 8973 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 2: 8844 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 1: 8839 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 0: 8782 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 3: 8965 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 1: 8739 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 2: 9006 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 0: 8892 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 3: 8968 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 1: 8891 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 2: 8887 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 0: 8766 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 3: 9032 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 1: 8866 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 2: 8933 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 0: 8852 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 3: 8895 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 2: 8895 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 1: 8798 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 3: 9082 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 2: 8927 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 0: 8806 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 1: 8801 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 2: 9056 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 0: 8922 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 3: 8939 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 1: 8854 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 2: 8936 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 0: 8765 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 1: 8848 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 3: 9039 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 0: 8872 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 2: 8951 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 1: 8768 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 3: 8957 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 0: 8793 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 2: 8935 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 1: 8773 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 3: 9111 GCS shadows traversed, 4001 replayed

Fri Nov 16 04:23:12 2012

LMS 2: 768 GCS shadows traversed, 334 replayed

Fri Nov 16 04:23:12 2012

LMS 0: 5407 GCS shadows traversed, 2454 replayed

Fri Nov 16 04:23:12 2012

LMS 1: 2724 GCS shadows traversed, 1235 replayed

Fri Nov 16 04:23:12 2012

LMS 3: 549 GCS shadows traversed, 247 replayed

Fri Nov 16 04:23:12 2012

Submitted all GCS remote-cache requests

Post SMON to start 1st pass IR

Fix write in gcs resources

Reconfiguration complete

Fri Nov 16 04:23:13 2012

parallel recovery started with 15 processes

Fri Nov 16 04:23:13 2012

Started redo scan

Fri Nov 16 04:23:13 2012

Completed redo scan

212 redo blocks read, 56 data blocks need recovery

Fri Nov 16 04:23:13 2012

Started redo application at

Thread 2: logseq 1954, block 119345

Fri Nov 16 04:23:13 2012

Recovery of Online Redo Log: Thread 2 Group 7 Seq 1954 Reading mem 0

Mem# 0 errs 0: &＃43;FOODSAFE_DISK/safedb/onlinelog/group_7.291.798134677

Fri Nov 16 04:23:13 2012

Completed redo application

Fri Nov 16 04:23:13 2012

Completed instance recovery at

Thread 2: logseq 1954, block 119557, scn 1438521085

53 data blocks read, 57 data blocks written, 212 redo blocks read

Switch log for thread 2 to sequence 1955

Fri Nov 16 04:23:13 2012

ARC3: Archiving disabled thread 2 sequence 1955

Fri Nov 16 04:23:18 2012

2012-11-16 12:44 上传

点击文件名下载附件

186.41 KB, 下载次数: 13