Hello, I am experiencing random reboots on random servers during the week at random time... I did some research and I am suspecting that the "automatic recovery for application health monitoring" is what causes that but I am not sure... Can any expert make a suggestion? I am attaching a copy of the log that is being generated during the shutdown...
00001028.0000367c::2014/02/25-06:12:34.316 INFO [RHS] Resource Virtual
Machine Configuration Computer-1 called SetResourceLockedMode.
LockedModeEnabled1, LockedModeReason0.
0000095c.00003230::2014/02/25-06:12:34.316 INFO [RCM] HandleMonitorReply:
LOCKEDMODE for 'Virtual Machine Configuration Computer-1', gen(0)
result 0/0.
0000095c.00003230::2014/02/25-06:12:34.316 INFO [RCM] Virtual Machine
Configuration Computer-1: Flags 1 added to StatusInformation. New
StatusInformation 1
00001028.0000367c::2014/02/25-06:12:34.316 INFO [RHS] Resource Virtual
Machine Computer-1 called SetResourceLockedMode. LockedModeEnabled1,
LockedModeReason0.
0000095c.00003230::2014/02/25-06:12:34.316 INFO [RCM] Computer-1:
Added Flags 1 to StatusInformation. New StatusInformation 1
0000095c.00001364::2014/02/25-06:12:34.316 INFO [GEM] Node 6: Sending 1
messages as a batched GEM message
0000095c.000020d0::2014/02/25-06:12:34.316 INFO [RCM] HandleMonitorReply:
LOCKEDMODE for 'Virtual Machine Computer-1', gen(0) result 0/0.
0000095c.000020d0::2014/02/25-06:12:34.316 INFO [RCM] Virtual Machine
Computer-1: Flags 1 added to StatusInformation. New StatusInformation 1
0000095c.00001364::2014/02/25-06:12:34.316 INFO [GEM] Node 6: Sending 1
messages as a batched GEM message
0000095c.000020d0::2014/02/25-06:12:34.320 INFO [GEM] Node 6: Sending 1
messages as a batched GEM message
0000095c.000020d0::2014/02/25-06:12:34.320 INFO [GUM] Node 6: Processing
RequestLock 6:321
0000095c.00000f30::2014/02/25-06:12:34.321 INFO [GUM] Node 6: Processing
GrantLock to 6 (sent by 3 gumid: 3827)
0000095c.000020d0::2014/02/25-06:12:34.321 INFO [GUM] Node 6: executing
request locally, gumId:3828, my action: /dm/update, # of updates: 1
0000095c.000020d0::2014/02/25-06:12:34.321 INFO [GEM] Node 6: Sending 1
messages as a batched GEM message
0000095c.00001b00::2014/02/25-06:12:34.323 INFO [RCM] HandleMonitorReply:
INMEMORY_NODELOCAL_PROPERTIES for 'Virtual Machine Computer-1', gen(0)
result 0/0.
0000095c.0000356c::2014/02/25-06:12:34.324 INFO [GEM] Node 6: Sending 1
messages as a batched GEM message
00001028.0000367c::2014/02/25-06:12:34.927 INFO [RHS] Resource Virtual
Machine Configuration Computer-1 called SetResourceLockedMode.
LockedModeEnabled0, LockedModeReason0.
0000095c.0000356c::2014/02/25-06:12:34.927 INFO [RCM] HandleMonitorReply:
LOCKEDMODE for 'Virtual Machine Configuration Computer-1', gen(0)
result 0/0.
0000095c.0000356c::2014/02/25-06:12:34.927 INFO [RCM] Virtual Machine
Configuration Computer-1: Flags 1 removed from StatusInformation. New
StatusInformation 0
00001028.0000367c::2014/02/25-06:12:34.928 INFO [RHS] Resource Virtual
Machine Computer-1 called SetResourceLockedMode. LockedModeEnabled0,
LockedModeReason0.
0000095c.00000140::2014/02/25-06:12:34.928 INFO [GEM] Node 6: Sending 1
messages as a batched GEM message
0000095c.00000c8c::2014/02/25-06:12:34.928 INFO [RCM] HandleMonitorReply:
LOCKEDMODE for 'Virtual Machine Computer-1', gen(0) result 0/0.
00001028.0000367c::2014/02/25-06:12:34.928 INFO [RES] Virtual Machine
<Virtual Machine Computer-1>: Current state 'Online', event 'VmStopped'
0000095c.00000c8c::2014/02/25-06:12:34.928 INFO [RCM] Virtual Machine
Computer-1: Flags 1 removed from StatusInformation. New
StatusInformation 0
0000095c.00000c8c::2014/02/25-06:12:34.928 INFO [RCM] Computer-1:
Removed Flags 1 from StatusInformation. New StatusInformation 0
0000095c.00000140::2014/02/25-06:12:34.928 INFO [GEM] Node 6: Sending 1
messages as a batched GEM message
00001028.0000367c::2014/02/25-06:12:34.928 INFO [RES] Virtual Machine
<Virtual Machine Computer-1>: State change 'Online' -> 'Offline'
0000095c.00000140::2014/02/25-06:12:34.928 INFO [GEM] Node 6: Sending 1
messages as a batched GEM message
0000095c.000017a8::2014/02/25-06:12:34.929 INFO [RCM]
rcm::RcmApi::OfflineResource: (Virtual Machine Computer-1, 1)
0000095c.000017a8::2014/02/25-06:12:34.929 INFO [GUM] Node 6: executing
request locally, gumId:3829, my action: /dm/update, # of updates: 1
0000095c.000017a8::2014/02/25-06:12:34.930 INFO [GEM] Node 6: Sending 1
messages as a batched GEM message
0000095c.000017a8::2014/02/25-06:12:34.931 INFO [RCM] Res Virtual Machine
Computer-1: Online -> WaitingToGoOffline( StateUnknown )
0000095c.000017a8::2014/02/25-06:12:34.931 INFO [RCM]
TransitionToState(Virtual Machine Computer-1)
Online-->WaitingToGoOffline.
0000095c.000017a8::2014/02/25-06:12:34.931 INFO [RCM]
rcm::RcmGroup::UpdateStateIfChanged: (Computer-1, Online --> Pending)
0000095c.000017a8::2014/02/25-06:12:34.931 INFO [RCM] Res Virtual Machine
Computer-1: WaitingToGoOffline -> OfflineCallIssued( StateUnknown )
0000095c.000017a8::2014/02/25-06:12:34.931 INFO [RCM]
TransitionToState(Virtual Machine Computer-1)
WaitingToGoOffline-->OfflineCallIssued.
0000095c.0000356c::2014/02/25-06:12:34.931 INFO [GEM] Node 6: Sending 1
messages as a batched GEM message
0000095c.00000de8::2014/02/25-06:12:34.931 INFO [RCM] ignored non-local
state Pending for group Computer-1
0000095c.0000356c::2014/02/25-06:12:34.931 INFO [GEM] Node 6: Sending 1
messages as a batched GEM message
00001028.000025cc::2014/02/25-06:12:34.931 INFO [RES] Virtual Machine
<Virtual Machine Computer-1>: Current state 'Offline', event 'Offline'
0000095c.0000356c::2014/02/25-06:12:34.932 INFO [RCM] HandleMonitorReply:
OFFLINERESOURCE for 'Virtual Machine Computer-1', gen(0) result 0/0.
0000095c.0000356c::2014/02/25-06:12:34.932 INFO [RCM] Res Virtual Machine
Computer-1: OfflineCallIssued -> OfflineSavingCheckpoints( StateUnknown
)
0000095c.0000356c::2014/02/25-06:12:34.932 INFO [RCM]
TransitionToState(Virtual Machine Computer-1)
OfflineCallIssued-->OfflineSavingCheckpoints.
0000095c.00000c8c::2014/02/25-06:12:34.932 INFO [RCM] Res Virtual Machine
Computer-1: OfflineSavingCheckpoints -> Offline( StateUnknown )
0000095c.00000c8c::2014/02/25-06:12:34.932 INFO [RCM]
TransitionToState(Virtual Machine Computer-1)
OfflineSavingCheckpoints-->Offline.
0000095c.00000c8c::2014/02/25-06:12:34.932 INFO [RCM]
rcm::RcmGroup::UpdateStateIfChanged: (Computer-1, Pending --> Offline)
0000095c.0000356c::2014/02/25-06:12:34.932 INFO [GEM] Node 6: Sending 1
messages as a batched GEM message
0000095c.00000c8c::2014/02/25-06:12:34.932 INFO [RCM] moved 0 tasks from
staging set to task set. TaskSetSize=0
0000095c.00000c8c::2014/02/25-06:12:34.932 INFO [RCM]
rcm::RcmPriorityManager::StartGroups: [RCM] done, executed 0 tasks
0000095c.0000356c::2014/02/25-06:12:34.932 INFO [GEM] Node 6: Sending 1
messages as a batched GEM message
0000095c.00000de8::2014/02/25-06:12:34.932 INFO [RCM] ignored non-local
state Offline for group Computer-1
0000095c.000017a8::2014/02/25-06:12:34.955 INFO [GUM] Node 6: executing
request locally, gumId:3830, my action: /dm/update, # of updates: 1
0000095c.000017a8::2014/02/25-06:12:34.955 INFO [GEM] Node 6: Sending 1
messages as a batched GEM message
0000095c.000017a8::2014/02/25-06:12:34.957 INFO [RCM] HandleMonitorReply:
INMEMORY_NODELOCAL_PROPERTIES for 'Virtual Machine Computer-1', gen(0)
result 0/0.
0000095c.000017a8::2014/02/25-06:12:34.957 INFO [GEM] Node 6: Sending 1
messages as a batched GEM message
0000095c.00000f30::2014/02/25-06:12:35.315 INFO [GEM] Node 6: Deleting
[3:1497 , 3:1497] (both included) as it has been ack'd by every node