We have setup 3 node hyper-v cluster and every two weeks one of the node will recover from the bug check(blue screen error) causing VM's to failover to other nodes or to be in hung state. I am suspecting the problem is with CSV. Please someone help on this.
below is the cluster log:
00000f08.0000494c::2016/07/20-20:59:28.664 WARN [RCM] ResourceTypeChaseTheOwnerLoop::DoCall: ResType MSMQ's DLL is not present on this node. Attempting to find a good node...
00000f08.0000494c::2016/07/20-20:59:28.680 WARN [RCM] ResourceTypeChaseTheOwnerLoop::DoCall: ResType MSMQ's DLL is not present on this node. Attempting to find a good node...
00000f08.0000494c::2016/07/20-20:59:28.696 WARN [RCM] ResourceTypeChaseTheOwnerLoop::DoCall: ResType MSMQTriggers's DLL is not present on this node. Attempting to find a good node...
00000f08.0000494c::2016/07/20-20:59:28.696 WARN [RCM] ResourceTypeChaseTheOwnerLoop::DoCall: ResType MSMQTriggers's DLL is not present on this node. Attempting to find a good node...
00001a24.00004d70::2016/07/20-20:59:48.421 INFO [RES] Physical Disk <Cluster Disk 3>: VolumeIsNtfs: Volume \\?\GLOBALROOT\Device\Harddisk2\ClusterPartition2\ has FS type NTFS
00000f08.00001ce8::2016/07/20-21:00:25.263 INFO [RCM [RES] Virtual Machine BHRMPRDPTAP2 embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00004738::2016/07/20-21:00:25.263 INFO [RCM [RES] Virtual Machine BHRMPRDSM embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00000ff8::2016/07/20-21:00:25.263 INFO [RCM [RES] Virtual Machine BHRMPRDPTCI embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.000001f8::2016/07/20-21:00:25.263 INFO [RCM [RES] Virtual Machine BHRMPRDBWDB embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.000028b4::2016/07/20-21:00:25.388 INFO [RCM [RES] Virtual Machine SAPRouter embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.000028b4::2016/07/20-21:00:25.388 INFO [RCM [RES] Virtual Machine BHRMPRDWD2 embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.000028b4::2016/07/20-21:00:25.388 INFO [RCM [RES] Virtual Machine BHRMPRDERPDB embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00001544::2016/07/21-01:33:14.358 INFO [RCM [RES] Virtual Machine BHRMPRDBWDB embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00001814::2016/07/21-01:33:14.358 INFO [RCM [RES] Virtual Machine BHRMPRDSM embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00004c2c::2016/07/21-01:33:14.358 INFO [RCM [RES] Virtual Machine BHRMPRDPTAP2 embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.000017ac::2016/07/21-01:33:14.358 INFO [RCM [RES] Virtual Machine BHRMPRDERPDB embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.0000224c::2016/07/21-01:33:14.358 INFO [RCM [RES] Virtual Machine BHRMPRDPTCI embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.000017ac::2016/07/21-01:33:14.483 INFO [RCM [RES] Virtual Machine BHRMPRDWD2 embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00002890::2016/07/21-01:33:14.483 INFO [RCM [RES] Virtual Machine SAPRouter embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
0000073c.00003cb0::2016/07/21-01:33:21.398 INFO [CAM] Substituting Token Owner: BUILTIN\Administrators, Original: NT AUTHORITY\SYSTEM
0000073c.00003cb0::2016/07/21-01:33:21.398 INFO [CAM] Token Created, Client Handle: 80006c7c
00001a24.00000e64::2016/07/21-01:33:50.909 INFO [RES] Physical Disk <Cluster Disk 3>: VolumeIsNtfs: Volume \\?\GLOBALROOT\Device\Harddisk2\ClusterPartition2\ has FS type NTFS
00000f08.00001814::2016/07/21-01:34:14.538 INFO [RCM [RES] Virtual Machine BHRMPRDPTAP2 embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.0000224c::2016/07/21-01:34:14.538 INFO [RCM [RES] Virtual Machine BHRMPRDSM embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00001544::2016/07/21-01:34:14.538 INFO [RCM [RES] Virtual Machine BHRMPRDBWDB embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.0000224c::2016/07/21-01:34:14.538 INFO [RCM [RES] Virtual Machine BHRMPRDERPDB embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00003300::2016/07/21-01:34:14.538 INFO [RCM [RES] Virtual Machine BHRMPRDPTCI embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00003300::2016/07/21-01:34:14.663 INFO [RCM [RES] Virtual Machine BHRMPRDWD2 embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00001544::2016/07/21-01:34:14.663 INFO [RCM [RES] Virtual Machine SAPRouter embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.0000224c::2016/07/21-01:34:31.770 INFO [DCM] HandleSweeperRecheck
00000f08.0000224c::2016/07/21-01:34:31.770 INFO [CLI] LsaCallAuthenticationPackage: 0, 0 size: 4, buffer: HDL( 842bef0000 )
00000f08.00000834::2016/07/21-01:34:31.770 INFO [DCM] HandleRequest: dcm/connectivityCheck
00000f08.0000224c::2016/07/21-01:34:31.770 INFO [DCM] SetVolumeMountPoint C:\ClusterStorage\Volume2\ => \\?\Volume{9e8c0ad8-b0dd-43e1-a1ea-686bdf4aa582}\
00000f08.0000224c::2016/07/21-01:34:31.770 INFO [DCM] SetVolumeMountPoint C:\ClusterStorage\Volume1\ => \\?\Volume{baf4e645-bc8b-4a6c-8881-98cbeb2df054}\
0000073c.00003cb0::2016/07/21-01:34:48.618 INFO [CAM] Substituting Token Owner: BUILTIN\Administrators, Original: NT AUTHORITY\SYSTEM
0000073c.00003cb0::2016/07/21-01:34:48.618 INFO [CAM] Token Created, Client Handle: 800064d0
00001a24.00003950::2016/07/21-01:34:50.920 INFO [RES] Physical Disk <Cluster Disk 3>: VolumeIsNtfs: Volume \\?\GLOBALROOT\Device\Harddisk2\ClusterPartition2\ has FS type NTFS
00000f08.00002890::2016/07/21-01:35:14.700 INFO [RCM [RES] Virtual Machine BHRMPRDPTAP2 embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00003300::2016/07/21-01:35:14.700 INFO [RCM [RES] Virtual Machine BHRMPRDBWDB embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00002890::2016/07/21-01:35:14.700 INFO [RCM [RES] Virtual Machine BHRMPRDSM embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.000017ac::2016/07/21-01:35:14.700 INFO [RCM [RES] Virtual Machine BHRMPRDPTCI embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.0000224c::2016/07/21-01:35:14.700 INFO [RCM [RES] Virtual Machine BHRMPRDERPDB embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00003300::2016/07/21-01:35:14.825 INFO [RCM [RES] Virtual Machine BHRMPRDWD2 embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.000017ac::2016/07/21-01:35:14.825 INFO [RCM [RES] Virtual Machine SAPRouter embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
0000073c.00004a64::2016/07/21-01:35:24.829 INFO [CAM] Substituting Token Owner: BUILTIN\Administrators, Original: NT AUTHORITY\SYSTEM
0000073c.00004a64::2016/07/21-01:35:24.829 INFO [CAM] Token Created, Client Handle: 80003168
00001a24.00002c0c::2016/07/21-01:35:50.931 INFO [RES] Physical Disk <Cluster Disk 3>: VolumeIsNtfs: Volume \\?\GLOBALROOT\Device\Harddisk2\ClusterPartition2\ has FS type NTFS
00000f08.000017ac::2016/07/21-01:36:14.869 INFO [RCM [RES] Virtual Machine BHRMPRDPTAP2 embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.0000224c::2016/07/21-01:36:14.869 INFO [RCM [RES] Virtual Machine BHRMPRDSM embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00002890::2016/07/21-01:36:14.869 INFO [RCM [RES] Virtual Machine BHRMPRDBWDB embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00002d2c::2016/07/21-01:36:14.869 INFO [RCM [RES] Virtual Machine BHRMPRDPTCI embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00002d2c::2016/07/21-01:36:14.869 INFO [RCM [RES] Virtual Machine BHRMPRDERPDB embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00001544::2016/07/21-01:36:14.978 INFO [RCM [RES] Virtual Machine SAPRouter embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00003300::2016/07/21-01:36:14.978 INFO [RCM [RES] Virtual Machine BHRMPRDWD2 embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00001a24.00002c0c::2016/07/21-01:36:50.934 INFO [RES] Physical Disk <Cluster Disk 3>: VolumeIsNtfs: Volume \\?\GLOBALROOT\Device\Harddisk2\ClusterPartition2\ has FS type NTFS
0000073c.00002970::2016/07/21-01:36:51.403 INFO [CAM] Substituting Token Owner: BUILTIN\Administrators, Original: NT AUTHORITY\SYSTEM
0000073c.00002970::2016/07/21-01:36:51.403 INFO [CAM] Token Created, Client Handle: 80003988
00000f08.00002890::2016/07/21-01:37:15.042 INFO [RCM [RES] Virtual Machine BHRMPRDBWDB embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00002d2c::2016/07/21-01:37:15.042 INFO [RCM [RES] Virtual Machine BHRMPRDPTAP2 embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00002d2c::2016/07/21-01:37:15.042 INFO [RCM [RES] Virtual Machine BHRMPRDSM embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00002890::2016/07/21-01:37:15.042 INFO [RCM [RES] Virtual Machine BHRMPRDERPDB embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00001544::2016/07/21-01:37:15.042 INFO [RCM [RES] Virtual Machine BHRMPRDPTCI embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00001544::2016/07/21-01:37:15.167 INFO [RCM [RES] Virtual Machine BHRMPRDWD2 embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.000017ac::2016/07/21-01:37:15.167 INFO [RCM [RES] Virtual Machine SAPRouter embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00001544::2016/07/21-01:37:31.825 INFO [DCM] HandleSweeperRecheck
00000f08.00001544::2016/07/21-01:37:31.825 INFO [CLI] LsaCallAuthenticationPackage: 0, 0 size: 4, buffer: HDL( 842bef0000 )
00000f08.00000834::2016/07/21-01:37:31.825 INFO [DCM] HandleRequest: dcm/connectivityCheck
00000f08.00001544::2016/07/21-01:37:31.825 INFO [DCM] SetVolumeMountPoint C:\ClusterStorage\Volume2\ => \\?\Volume{9e8c0ad8-b0dd-43e1-a1ea-686bdf4aa582}\
00000f08.00001544::2016/07/21-01:37:31.825 INFO [DCM] SetVolumeMountPoint C:\ClusterStorage\Volume1\ => \\?\Volume{baf4e645-bc8b-4a6c-8881-98cbeb2df054}\
00001a24.00002c0c::2016/07/21-01:37:50.945 INFO [RES] Physical Disk <Cluster Disk 3>: VolumeIsNtfs: Volume \\?\GLOBALROOT\Device\Harddisk2\ClusterPartition2\ has FS type NTFS
00000f08.000017ac::2016/07/21-01:38:08.182 INFO [RCM] [GIM] Scheduling Local Node Crawler to run in 300000 millisec.
00000f08.00001544::2016/07/21-01:38:15.207 INFO [RCM [RES] Virtual Machine BHRMPRDPTAP2 embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00001544::2016/07/21-01:38:15.207 INFO [RCM [RES] Virtual Machine BHRMPRDPTCI embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00002890::2016/07/21-01:38:15.207 INFO [RCM [RES] Virtual Machine BHRMPRDSM embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00003a08::2016/07/21-01:38:15.207 INFO [RCM [RES] Virtual Machine BHRMPRDBWDB embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00003a08::2016/07/21-01:38:15.207 INFO [RCM [RES] Virtual Machine BHRMPRDERPDB embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00003a08::2016/07/21-01:38:15.332 INFO [RCM [RES] Virtual Machine BHRMPRDWD2 embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00000f08.00001814::2016/07/21-01:38:15.332 INFO [RCM [RES] Virtual Machine SAPRouter embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2
00001a24.00002e54::2016/07/21-01:38:50.948 INFO [RES] Physical Disk <Cluster Disk 3>: VolumeIsNtfs: Volume \\?\GLOBALROOT\Device\Harddisk2\ClusterPartition2\ has FS type NTFS
00001a24.00003698::2016/07/21-01:39:50.958 INFO [RES] Physical Disk <Cluster Disk 3>: VolumeIsNtfs: Volume \\?\GLOBALROOT\Device\Harddisk2\ClusterPartition2\ has FS type NTFS
00000f08.00003a08::2016/07/21-01:40:31.872 INFO [DCM] HandleSweeperRecheck
00000f08.00003a08::2016/07/21-01:40:31.872 INFO [CLI] LsaCallAuthenticationPackage: 0, 0 size: 4, buffer: HDL( 842bf70000 )
00000f08.00000834::2016/07/21-01:40:31.872 INFO [DCM] HandleRequest: dcm/connectivityCheck
00000f08.00003a08::2016/07/21-01:40:31.872 INFO [DCM] SetVolumeMountPoint C:\ClusterStorage\Volume2\ => \\?\Volume{9e8c0ad8-b0dd-43e1-a1ea-686bdf4aa582}\
00000f08.00003a08::2016/07/21-01:40:31.872 INFO [DCM] SetVolumeMountPoint C:\ClusterStorage\Volume1\ => \\?\Volume{baf4e645-bc8b-4a6c-8881-98cbeb2df054}\
00001a24.000049c4::2016/07/21-01:40:50.965 INFO [RES] Physical Disk <Cluster Disk 3>: VolumeIsNtfs: Volume \\?\GLOBALROOT\Device\Harddisk2\ClusterPartition2\ has FS type NTFS
00001a24.00001984::2016/07/21-01:41:50.979 INFO [RES] Physical Disk <Cluster Disk 3>: VolumeIsNtfs: Volume \\?\GLOBALROOT\Device\Harddisk2\ClusterPartition2\ has FS type NTFS
00001a24.000010f0::2016/07/21-01:42:50.984 INFO [RES] Physical Disk <Cluster Disk 3>: VolumeIsNtfs: Volume \\?\GLOBALROOT\Device\Harddisk2\ClusterPartition2\ has FS type NTFS
00000f08.00001804::2016/07/21-01:42:58.908 INFO [GEM] Node 1: Deleting [2:46817 , 2:46835] (both included) as it has been ack'd by every node
0000073c.00002abc::2016/07/21-01:43:11.685 INFO [CAM] Token Created, Client Handle: 80004fc8
00000f08.00004b08::2016/07/21-01:43:12.733 INFO [GUM] Node 1: Executing locally gumId: 20020, updates: 1, first action: /dm/update
0000073c.00003e78::2016/07/21-01:43:13.029 INFO [CAM] Token Created, Client Handle: 80003120
00000f08.0000485c::2016/07/21-01:43:13.725 INFO [GEM] Node 1: Deleting [3:3415 , 3:3415] (both included) as it has been ack'd by every node
00000f08.00001804::2016/07/21-01:43:14.069 INFO [GUM] Node 1: Processing RequestLock 2:1551
00000f08.0000485c::2016/07/21-01:43:14.069 INFO [GUM] Node 1: Processing GrantLock to 2 (sent by 3 gumid: 20020)
00000f08.00001814::2016/07/21-01:43:14.069 INFO [GUM] Node 1: Executing locally gumId: 20021, updates: 1, first action: /dm/update
00000f08.00001804::2016/07/21-01:43:15.069 INFO [GEM] Node 1: Deleting [2:46836 , 2:46837] (both included) as it has been ack'd by every node
00000f08.0000485c::2016/07/21-01:43:15.069 INFO [GEM] Node 1: Deleting [3:3416 , 3:3416] (both included) as it has been ack'd by every node
00000f08.00002d2c::2016/07/21-01:43:31.927 INFO [DCM] HandleSweeperRecheck
00000f08.00002d2c::2016/07/21-01:43:31.927 INFO [CLI] LsaCallAuthenticationPackage: 0, 0 size: 4, buffer: HDL( 842c430000 )
00000f08.00000834::2016/07/21-01:43:31.927 INFO [DCM] HandleRequest: dcm/connectivityCheck
00000f08.00002d2c::2016/07/21-01:43:31.927 INFO [DCM] SetVolumeMountPoint C:\ClusterStorage\Volume2\ => \\?\Volume{9e8c0ad8-b0dd-43e1-a1ea-686bdf4aa582}\
00000f08.00002d2c::2016/07/21-01:43:31.927 INFO [DCM] SetVolumeMountPoint C:\ClusterStorage\Volume1\ => \\?\Volume{baf4e645-bc8b-4a6c-8881-98cbeb2df054}\
00001a24.000010f0::2016/07/21-01:43:50.993 INFO [RES] Physical Disk <Cluster Disk 3>: VolumeIsNtfs: Volume \\?\GLOBALROOT\Device\Harddisk2\ClusterPartition2\ has FS type NTFS
00001a24.00003f4c::2016/07/21-01:44:51.002 INFO [RES] Physical Disk <Cluster Disk 3>: VolumeIsNtfs: Volume \\?\GLOBALROOT\Device\Harddisk2\ClusterPartition2\ has FS type NTFS
00001a24.00002948::2016/07/21-01:45:51.005 INFO [RES] Physical Disk <Cluster Disk 3>: VolumeIsNtfs: Volume \\?\GLOBALROOT\Device\Harddisk2\ClusterPartition2\ has FS type NTFS
00001b24.00001b48::2016/07/21-01:46:15.342 ERR [RHS] RhsCall::DeadlockMonitor: Call ISALIVE timed out by 15 milliseconds for resource 'Virtual Machine BHRMPRDBWDB'.
00001cec.00001d10::2016/07/21-01:46:15.342 ERR [RHS] RhsCall::DeadlockMonitor: Call ISALIVE timed out by 15 milliseconds for resource 'Virtual Machine BHRMPRDPTAP2'.
00001d18.00001d3c::2016/07/21-01:46:15.342 ERR [RHS] RhsCall::DeadlockMonitor: Call ISALIVE timed out by 15 milliseconds for resource 'Virtual Machine BHRMPRDPTCI'.
0000182c.00001ab0::2016/07/21-01:46:15.342 ERR [RHS] RhsCall::DeadlockMonitor: Call ISALIVE timed out by 15 milliseconds for resource 'Virtual Machine BHRMPRDERPDB'.
00001d70.00001d94::2016/07/21-01:46:15.342 ERR [RHS] RhsCall::DeadlockMonitor: Call ISALIVE timed out by 15 milliseconds for resource 'Virtual Machine BHRMPRDSM'.
0000182c.00001ab0::2016/07/21-01:46:15.342 INFO [RHS] Enabling RHS termination watchdog with timeout 1680000 and recovery action 3 from source 5.
0000182c.00001ab0::2016/07/21-01:46:15.342 ERR [RHS] Resource Virtual Machine BHRMPRDERPDB handling deadlock. Cleaning current operation and terminating RHS process.
00001d18.00001d3c::2016/07/21-01:46:15.342 INFO [RHS] Enabling RHS termination watchdog with timeout 1680000 and recovery action 3 from source 5.
00001d18.00001d3c::2016/07/21-01:46:15.342 ERR [RHS] Resource Virtual Machine BHRMPRDPTCI handling deadlock. Cleaning current operation and terminating RHS process.
00001d70.00001d94::2016/07/21-01:46:15.342 INFO [RHS] Enabling RHS termination watchdog with timeout 1680000 and recovery action 3 from source 5.
00001b24.00001b48::2016/07/21-01:46:15.342 INFO [RHS] Enabling RHS termination watchdog with timeout 1680000 and recovery action 3 from source 5.
00001d70.00001d94::2016/07/21-01:46:15.342 ERR [RHS] Resource Virtual Machine BHRMPRDSM handling deadlock. Cleaning current operation and terminating RHS process.
00001cec.00001d10::2016/07/21-01:46:15.342 INFO [RHS] Enabling RHS termination watchdog with timeout 1680000 and recovery action 3 from source 5.
00001b24.00001b48::2016/07/21-01:46:15.342 ERR [RHS] Resource Virtual Machine BHRMPRDBWDB handling deadlock. Cleaning current operation and terminating RHS process.
00001cec.00001d10::2016/07/21-01:46:15.342 ERR [RHS] Resource Virtual Machine BHRMPRDPTAP2 handling deadlock. Cleaning current operation and terminating RHS process.
00000f08.00003a08::2016/07/21-01:46:15.342 WARN [RCM] HandleMonitorReply: FAILURENOTIFICATION for 'Virtual Machine BHRMPRDERPDB', gen(0) result 4/0.
00000f08.0000224c::2016/07/21-01:46:15.342 WARN [RCM] HandleMonitorReply: FAILURENOTIFICATION for 'Virtual Machine BHRMPRDPTCI', gen(0) result 4/0.
00001d18.00001d3c::2016/07/21-01:46:15.342 ERR [RHS] About to send WER report.
00000f08.00003a08::2016/07/21-01:46:15.342 INFO [RCM] rcm::RcmResource::HandleMonitorReply: Resource 'Virtual Machine BHRMPRDERPDB' consecutive failure count 1.
0000182c.00001ab0::2016/07/21-01:46:15.342 ERR [RHS] About to send WER report.
00000f08.0000224c::2016/07/21-01:46:15.342 INFO [RCM] rcm::RcmResource::HandleMonitorReply: Resource 'Virtual Machine BHRMPRDPTCI' consecutive failure count 1.
00001d70.00001d94::2016/07/21-01:46:15.342 ERR [RHS] About to send WER report.
00000f08.00002d2c::2016/07/21-01:46:15.342 WARN [RCM] HandleMonitorReply: FAILURENOTIFICATION for 'Virtual Machine BHRMPRDSM', gen(0) result 4/0.
00000f08.00003a08::2016/07/21-01:46:15.342 WARN [RCM] HandleMonitorReply: FAILURENOTIFICATION for 'Virtual Machine BHRMPRDBWDB', gen(0) result 4/0.
00000f08.00002d2c::2016/07/21-01:46:15.342 INFO [RCM] rcm::RcmResource::HandleMonitorReply: Resource 'Virtual Machine BHRMPRDSM' consecutive failure count 1.
00000f08.00003a08::2016/07/21-01:46:15.342 INFO [RCM] rcm::RcmResource::HandleMonitorReply: Resource 'Virtual Machine BHRMPRDBWDB' consecutive failure count 1.
00001b24.00001b48::2016/07/21-01:46:15.342 ERR [RHS] About to send WER report.
00001cec.00001d10::2016/07/21-01:46:15.342 ERR [RHS] About to send WER report.
00000f08.000017ac::2016/07/21-01:46:15.342 WARN [RCM] HandleMonitorReply: FAILURENOTIFICATION for 'Virtual Machine BHRMPRDPTAP2', gen(0) result 4/0.
00000f08.000017ac::2016/07/21-01:46:15.342 INFO [RCM] rcm::RcmResource::HandleMonitorReply: Resource 'Virtual Machine BHRMPRDPTAP2' consecutive failure count 1.
00001900.00001924::2016/07/21-01:46:15.452 ERR [RHS] RhsCall::DeadlockMonitor: Call ISALIVE timed out by 16 milliseconds for resource 'Virtual Machine SAPRouter'.
00001900.00001924::2016/07/21-01:46:15.452 INFO [RHS] Enabling RHS termination watchdog with timeout 1680000 and recovery action 3 from source 5.
00001900.00001924::2016/07/21-01:46:15.452 ERR [RHS] Resource Virtual Machine SAPRouter handling deadlock. Cleaning current operation and terminating RHS process.
00000f08.000017ac::2016/07/21-01:46:15.452 WARN [RCM] HandleMonitorReply: FAILURENOTIFICATION for 'Virtual Machine SAPRouter', gen(0) result 4/0.
00001900.00001924::2016/07/21-01:46:15.452 ERR [RHS] About to send WER report.
00000f08.000017ac::2016/07/21-01:46:15.452 INFO [RCM] rcm::RcmResource::HandleMonitorReply: Resource 'Virtual Machine SAPRouter' consecutive failure count 1.
00001dcc.00001df0::2016/07/21-01:46:15.467 ERR [RHS] RhsCall::DeadlockMonitor: Call ISALIVE timed out by 15 milliseconds for resource 'Virtual Machine BHRMPRDWD2'.
00001dcc.00001df0::2016/07/21-01:46:15.467 INFO [RHS] Enabling RHS termination watchdog with timeout 1680000 and recovery action 3 from source 5.
00001dcc.00001df0::2016/07/21-01:46:15.467 ERR [RHS] Resource Virtual Machine BHRMPRDWD2 handling deadlock. Cleaning current operation and terminating RHS process.
00000f08.000017ac::2016/07/21-01:46:15.467 WARN [RCM] HandleMonitorReply: FAILURENOTIFICATION for 'Virtual Machine BHRMPRDWD2', gen(0) result 4/0.
00000f08.000017ac::2016/07/21-01:46:15.467 INFO [RCM] rcm::RcmResource::HandleMonitorReply: Resource 'Virtual Machine BHRMPRDWD2' consecutive failure count 1.
00001dcc.00001df0::2016/07/21-01:46:15.467 ERR [RHS] About to send WER report.
00000f08.000017ac::2016/07/21-01:46:31.970 INFO [DCM] HandleSweeperRecheck
00000f08.000017ac::2016/07/21-01:46:31.970 INFO [CLI] LsaCallAuthenticationPackage: 0, 0 size: 4, buffer: HDL( 842c430000 )
00000f08.00000834::2016/07/21-01:46:31.970 INFO [DCM] HandleRequest: dcm/connectivityCheck
00000f08.000017ac::2016/07/21-01:46:31.970 INFO [DCM] SetVolumeMountPoint C:\ClusterStorage\Volume2\ => \\?\Volume{9e8c0ad8-b0dd-43e1-a1ea-686bdf4aa582}\
00000f08.000017ac::2016/07/21-01:46:31.970 INFO [DCM] SetVolumeMountPoint C:\ClusterStorage\Volume1\ => \\?\Volume{baf4e645-bc8b-4a6c-8881-98cbeb2df054}\
00001a24.00004a18::2016/07/21-01:46:51.018 INFO [RES] Physical Disk <Cluster Disk 3>: VolumeIsNtfs: Volume \\?\GLOBALROOT\Device\Harddisk2\ClusterPartition2\ has FS type NTFS
00001a24.00002dec::2016/07/21-01:47:51.030 INFO [RES] Physical Disk <Cluster Disk 3>: VolumeIsNtfs: Volume \\?\GLOBALROOT\Device\Harddisk2\ClusterPartition2\ has FS type NTFS
00000f08.00002914::2016/07/21-01:48:12.877 ERR [STM] mscs::stm::STM::ProcessMessageWorker@610 had !ERROR! 5910
000018f8.0000192c::2016/07/21-01:48:12.893 ERR [RHS] RhsCall::DeadlockMonitor: Call RESOURCETYPECONTROL timed out by 16 milliseconds for resource '<none>'.
000018f8.0000192c::2016/07/21-01:48:12.893 INFO [RHS] Enabling RHS termination watchdog with timeout 1200000 and recovery action 3 from source 6.
000018f8.0000192c::2016/07/21-01:48:12.893 ERR [RHS] Resource Type Storage Pool handling deadlock. Cleaning current operation and terminating RHS process.
000018f8.0000192c::2016/07/21-01:48:12.893 ERR [RHS] About to send WER report.
00000f08.00004524::2016/07/21-01:48:14.209 ERR [STM] mscs::stm::STM::ProcessMessageWorker@610 had !ERROR! 5910
00001a24.00004290::2016/07/21-01:48:51.031 INFO [RES] Physical Disk <Cluster Disk 3>: VolumeIsNtfs: Volume \\?\GLOBALROOT\Device\Harddisk2\ClusterPartition2\ has FS type NTFS
00000f08.00001814::2016/07/21-01:49:32.017 INFO [DCM] HandleSweeperRecheck
00000f08.00001814::2016/07/21-01:49:32.017 INFO [CLI] LsaCallAuthenticationPackage: 0, 0 size: 4, buffer: HDL( 842c430000 )
00000f08.00000834::2016/07/21-01:49:32.017 INFO [DCM] HandleRequest: dcm/connectivityCheck
00000f08.00001814::2016/07/21-01:49:32.017 INFO [DCM] SetVolumeMountPoint C:\ClusterStorage\Volume2\ => \\?\Volume{9e8c0ad8-b0dd-43e1-a1ea-686bdf4aa582}\
00000f08.00001814::2016/07/21-01:49:32.017 INFO [DCM] SetVolumeMountPoint C:\ClusterStorage\Volume1\ => \\?\Volume{baf4e645-bc8b-4a6c-8881-98cbeb2df054}\
00000eec.00000ef0::2016/07/21-02:55:40.273 INFO -----------------------------+ LOG BEGIN +-----------------------------
00000eec.00000ef0::2016/07/21-02:55:40.273 INFO [CS] Starting clussvc as a service
00000eec.00000ef0::2016/07/21-02:55:40.273 INFO [CS] cluster service logging level is 5
00000eec.00000f34::2016/07/21-02:55:40.273 INFO [CS] Creating cluster node <vector len='1'>
00000eec.00000f34::2016/07/21-02:55:40.273 INFO <item>ClusSvc</item>
00000eec.00000f34::2016/07/21-02:55:40.273 INFO </vector>
00000eec.00000f44::2016/07/21-02:55:40.321 INFO [StartupConfig]: Initializing.
---------------------------------------------------------------------------------------------------------------------------------------------------------