Quantcast
Channel: High Availability (Clustering) forum
Viewing all articles
Browse latest Browse all 5654
↧

Microsoft Failover Clustering Events 1126, 1127 and 1129. Hearbeat Network Deprecated

August 25, 2015, 10:13 am
$
0
0

Environment and problem:

2 servers 2012 in a Microsoft failover clustering which is going to be used for SQL AlwaysON. There is no common storage between the servers, since I do not need one. These are both virtual servers running on ESXi environment 5.5

The servers are using a heartbeat network - 10.0.160.51 (node A) and 10.0.160.52 (node B) . Servers are across datacenters and we are using OTV to extend the heartbeat network from one site to another. The cluster validation report is all Green. But I am getting errors for heartbeat network cannot be reached by another node and heartbeat network partitioned errors - Event ID 1126, 1127 And 1129.

The errors are coming every 10-15 min interval. Here is the cluster.log events at one particular time when I got those errors (11:33:49 and 11:33:54 am - 8/25/2015)

000027b8.0000212c::2015/08/25-11:33:00.029 INFO  [NM] Received request from client address TVPPSQLTCMW01A.
000027b8.0000212c::2015/08/25-11:33:04.435 INFO  [NM] Received request from client address TVPPSQLTCMW01A.
000027b8.000021b0::2015/08/25-11:33:49.749 DBG   [NETFTAPI] Signaled NetftRemoteUnreachable event, local address 10.0.160.51:3343 remote address 10.0.160.52:3343
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  [IM] got event: Remote endpoint 10.0.160.52:~3343~ unreachable from 10.0.160.51:~3343~
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  [IM] Marking Route from 10.0.160.51:~3343~ to 10.0.160.52:~3343~ as down
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  [NDP] Checking to see if all routes for route (virtual) local fe80::8cce:1c3c:9d53:51cd:~0~ to remote fe80::805a:5752:fc4a:de9b:~0~ are down
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  [NDP] Route local 10.11.137.51:~0~ to remote 10.11.137.52:~0~ is up
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  [IM] Route history 1: Old: 00.921, Message: Response, Route sequence: 79338, Received sequence: 79338, Heartbeats counter/threshold: 5/5, Error: Success, NtStatus: 0 Timestamp: 2015/08/25-11:33:48.827, Ticks since last sending: 0
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  [IM] Route history 2: Old: 00.921, Message: Request, Route sequence: 79338, Received sequence: 79338, Heartbeats counter/threshold: 5/5, Error: Success, NtStatus: 0 Timestamp: 2015/08/25-11:33:48.827, Ticks since last sending: 0
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  [IM] Route history 3: Old: 00.921, Message: Response, Route sequence: 79338, Received sequence: 79337, Heartbeats counter/threshold: 5/5, Error: Success, NtStatus: 0 Timestamp: 2015/08/25-11:33:48.827, Ticks since last sending: 0
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  [IM] Route history 4: Old: 01.921, Message: Request, Route sequence: 79337, Received sequence: 79337, Heartbeats counter/threshold: 5/5, Error: Success, NtStatus: 0 Timestamp: 2015/08/25-11:33:47.827, Ticks since last sending: 0
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  [IM] Route history 5: Old: 01.921, Message: Response, Route sequence: 79337, Received sequence: 79336, Heartbeats counter/threshold: 5/5, Error: Success, NtStatus: 0 Timestamp: 2015/08/25-11:33:47.827, Ticks since last sending: 0
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  [IM] Route history 6: Old: 02.921, Message: Request, Route sequence: 79336, Received sequence: 79336, Heartbeats counter/threshold: 5/5, Error: Success, NtStatus: 0 Timestamp: 2015/08/25-11:33:46.827, Ticks since last sending: 0
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  [IM] Route history 7: Old: 02.921, Message: Response, Route sequence: 79336, Received sequence: 79335, Heartbeats counter/threshold: 5/5, Error: Success, NtStatus: 0 Timestamp: 2015/08/25-11:33:46.827, Ticks since last sending: 0
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  [IM] Route history 8: Old: 03.921, Message: Request, Route sequence: 79335, Received sequence: 79335, Heartbeats counter/threshold: 5/5, Error: Success, NtStatus: 0 Timestamp: 2015/08/25-11:33:45.827, Ticks since last sending: 0
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  [IM] Route history 9: Old: 03.921, Message: Response, Route sequence: 79335, Received sequence: 79334, Heartbeats counter/threshold: 5/5, Error: Success, NtStatus: 0 Timestamp: 2015/08/25-11:33:45.827, Ticks since last sending: 0
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  [IM] Route history 10: Old: 04.921, Message: Request, Route sequence: 79334, Received sequence: 79334, Heartbeats counter/threshold: 5/5, Error: Success, NtStatus: 0 Timestamp: 2015/08/25-11:33:44.827, Ticks since last sending: 0
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  [IM] Adding information for route Route from local 10.0.160.51:~3343~ to remote 10.0.160.52:~3343~, status: false, attributes: 0
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  [IM] Sending connectivity report to leader (node 1): <class mscs::InterfaceReport>
000027b8.00001f00::2015/08/25-11:33:49.749 INFO    <fromInterface>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</fromInterface>
000027b8.00001f00::2015/08/25-11:33:49.749 INFO    <upInterfaces><vector len='1'>
000027b8.00001f00::2015/08/25-11:33:49.749 INFO      <item>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</item>
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  </vector>
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  </upInterfaces>
000027b8.00001f00::2015/08/25-11:33:49.749 INFO    <downInterfaces><vector len='1'>
000027b8.00001f00::2015/08/25-11:33:49.749 INFO      <item>5f354f39-8e3b-4ad1-bb13-96e2e1d75d55</item>
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  </vector>
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  </downInterfaces>
000027b8.00001f00::2015/08/25-11:33:49.749 INFO    <upRoutesType><vector len='0'>
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  </vector>
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  </upRoutesType>
000027b8.00001f00::2015/08/25-11:33:49.749 INFO    <downRoutesType><vector len='1'>
000027b8.00001f00::2015/08/25-11:33:49.749 INFO      <item>1</item>
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  </vector>
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  </downRoutesType>
000027b8.00001f00::2015/08/25-11:33:49.749 INFO    <viewId>201</viewId>
000027b8.00001f00::2015/08/25-11:33:49.749 INFO    <localDisconnect>false</localDisconnect>
000027b8.00001f00::2015/08/25-11:33:49.749 INFO  </class mscs::InterfaceReport>
000027b8.0000212c::2015/08/25-11:33:49.749 INFO  [DCM] HandleNetftRemoteRouteChange
000027b8.00002438::2015/08/25-11:33:49.749 INFO  [IM] Leader got report from 1
000027b8.000013c4::2015/08/25-11:33:49.749 INFO  [DCM] HandleRequest: dcm/netftRouteChange
000027b8.00002438::2015/08/25-11:33:49.749 INFO  [IM - Heartbeat Network] 1 reports in state calculator queue
000027b8.00002410::2015/08/25-11:33:49.749 INFO  [IM - Heartbeat Network] State calculator got new report from 5ced600f-83b3-4fde-aa4e-f3d2e4f6e584
000027b8.000013c4::2015/08/25-11:33:49.749 INFO  [DCM] Forcing disconnect succeeded
000027b8.00002410::2015/08/25-11:33:49.749 INFO  [IM - Heartbeat Network] 0 reports in state calculator queue
000027b8.000013c4::2015/08/25-11:33:49.749 INFO  [DCM] Skipping client access network d7dcde53-bf75-427e-8083-c2f4166be39c for multichannel
000027b8.000013c4::2015/08/25-11:33:49.749 WARN  [DCM] No matching addresses for Netft on this node with id 1
000027b8.000013c4::2015/08/25-11:33:49.749 INFO  [DCM] Unregistering name fe80::8cce:1c3c:9d53:51cd for multichannel support returned 0
000027b8.0000212c::2015/08/25-11:33:49.764 DBG   [NETFTAPI] received NsiParameterNotification for 10.0.160.51 (IpDadStateDeprecated)
000027b8.0000212c::2015/08/25-11:33:49.764 DBG   [NETFTAPI] Signaled NetftLocalDisconnect event for 10.0.160.51
000027b8.00001f00::2015/08/25-11:33:49.764 INFO  [IM] got event: Local endpoint 10.0.160.51:~0~ disconnected
000027b8.00001f00::2015/08/25-11:33:49.764 INFO  [IM] Informing leader about local disconnect. Endpoint 10.0.160.51:~0~ is disconnected
000027b8.00001f00::2015/08/25-11:33:49.764 INFO  [IM] Adding information for route Route from local 10.0.160.51:~3343~ to remote 10.0.160.52:~3343~, status: false, attributes: 0
000027b8.00001f00::2015/08/25-11:33:49.764 INFO  [IM] Sending connectivity report to leader (node 1): <class mscs::InterfaceReport>
000027b8.00001f00::2015/08/25-11:33:49.764 INFO    <fromInterface>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</fromInterface>
000027b8.00001f00::2015/08/25-11:33:49.764 INFO    <upInterfaces><vector len='0'>
000027b8.00001f00::2015/08/25-11:33:49.764 INFO  </vector>
000027b8.00001f00::2015/08/25-11:33:49.764 INFO  </upInterfaces>
000027b8.00001f00::2015/08/25-11:33:49.764 INFO    <downInterfaces><vector len='2'>
000027b8.00001f00::2015/08/25-11:33:49.764 INFO      <item>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</item>
000027b8.00001f00::2015/08/25-11:33:49.764 INFO      <item>5f354f39-8e3b-4ad1-bb13-96e2e1d75d55</item>
000027b8.00001f00::2015/08/25-11:33:49.764 INFO  </vector>
000027b8.00001f00::2015/08/25-11:33:49.764 INFO  </downInterfaces>
000027b8.00001f00::2015/08/25-11:33:49.764 INFO    <upRoutesType><vector len='0'>
000027b8.00001f00::2015/08/25-11:33:49.764 INFO  </vector>
000027b8.00001f00::2015/08/25-11:33:49.764 INFO  </upRoutesType>
000027b8.00001f00::2015/08/25-11:33:49.764 INFO    <downRoutesType><vector len='1'>
000027b8.00001f00::2015/08/25-11:33:49.764 INFO      <item>1</item>
000027b8.00001f00::2015/08/25-11:33:49.764 INFO  </vector>
000027b8.00001f00::2015/08/25-11:33:49.764 INFO  </downRoutesType>
000027b8.00001f00::2015/08/25-11:33:49.764 INFO    <viewId>201</viewId>
000027b8.00001f00::2015/08/25-11:33:49.764 INFO    <localDisconnect>true</localDisconnect>
000027b8.00001f00::2015/08/25-11:33:49.764 INFO  </class mscs::InterfaceReport>
000027b8.00002438::2015/08/25-11:33:49.764 INFO  [IM] Leader got report from 1
000027b8.00002438::2015/08/25-11:33:49.764 INFO  [IM - Heartbeat Network] 1 reports in state calculator queue
000027b8.00002410::2015/08/25-11:33:49.764 INFO  [IM - Heartbeat Network] State calculator got new report from 5ced600f-83b3-4fde-aa4e-f3d2e4f6e584
000027b8.00002410::2015/08/25-11:33:49.764 INFO  [IM - Heartbeat Network] Issuing state change update with result <class mscs::InterfaceResult>
000027b8.00002410::2015/08/25-11:33:49.764 INFO    <up><vector len='0'>
000027b8.00002410::2015/08/25-11:33:49.764 INFO  </vector>
000027b8.00002410::2015/08/25-11:33:49.764 INFO  </up>
000027b8.00002410::2015/08/25-11:33:49.764 INFO    <down><vector len='1'>
000027b8.00002410::2015/08/25-11:33:49.764 INFO      <item>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</item>
000027b8.00002410::2015/08/25-11:33:49.764 INFO  </vector>
000027b8.00002410::2015/08/25-11:33:49.764 INFO  </down>
000027b8.00002410::2015/08/25-11:33:49.764 INFO    <unreachable><vector len='0'>
000027b8.00002410::2015/08/25-11:33:49.764 INFO  </vector>
000027b8.00002410::2015/08/25-11:33:49.764 INFO  </unreachable>
000027b8.00002410::2015/08/25-11:33:49.764 INFO  </class mscs::InterfaceResult>
000027b8.00002410::2015/08/25-11:33:49.764 INFO  [GEM] Node 1: Sending 1 messages as a batched GEM message
000027b8.000027bc::2015/08/25-11:33:49.764 INFO  [IM] Changing the state of adapters according to result: <class mscs::InterfaceResult>
000027b8.000027bc::2015/08/25-11:33:49.764 INFO    <up><vector len='0'>
000027b8.000027bc::2015/08/25-11:33:49.764 INFO  </vector>
000027b8.000027bc::2015/08/25-11:33:49.764 INFO  </up>
000027b8.000027bc::2015/08/25-11:33:49.764 INFO    <down><vector len='1'>
000027b8.000027bc::2015/08/25-11:33:49.764 INFO      <item>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</item>
000027b8.000027bc::2015/08/25-11:33:49.764 INFO  </vector>
000027b8.000027bc::2015/08/25-11:33:49.764 INFO  </down>
000027b8.000027bc::2015/08/25-11:33:49.764 INFO    <unreachable><vector len='0'>
000027b8.000027bc::2015/08/25-11:33:49.764 INFO  </vector>
000027b8.000027bc::2015/08/25-11:33:49.764 INFO  </unreachable>
000027b8.000027bc::2015/08/25-11:33:49.764 INFO  </class mscs::InterfaceResult>
000027b8.00002a64::2015/08/25-11:33:49.764 INFO  [DCM] HandleInterfaceChange
000027b8.000013c4::2015/08/25-11:33:49.764 INFO  [DCM] HandleRequest: dcm/connectivityCheck
000027b8.00002410::2015/08/25-11:33:49.764 INFO  [IM - Heartbeat Network] 0 reports in state calculator queue
000027b8.000013c4::2015/08/25-11:33:49.764 INFO  [DCM] Skipping client access network d7dcde53-bf75-427e-8083-c2f4166be39c for multichannel
000027b8.000013c4::2015/08/25-11:33:49.764 INFO  [DCM] Sending local node id to node Id 2
000027b8.00002ddc::2015/08/25-11:33:49.764 DBG   [NETFTAPI] received NsiDeleteInstance for fe80::5efe:10.0.160.51
000027b8.00002ddc::2015/08/25-11:33:49.780 WARN  [NETFTAPI] Failed to query parameters for fe80::5efe:10.0.160.51 (status 0x80070490)
000027b8.00002ddc::2015/08/25-11:33:49.780 DBG   [NETFTAPI] Signaled NetftLocalAdd event for fe80::5efe:10.0.160.51
000027b8.00002ddc::2015/08/25-11:33:49.796 WARN  [NETFTAPI] Failed to query parameters for fe80::5efe:10.0.160.51 (status 0x80070490)
000027b8.00002ddc::2015/08/25-11:33:49.796 DBG   [NETFTAPI] Signaled NetftLocalRemove event for fe80::5efe:10.0.160.51
000027b8.00002448::2015/08/25-11:33:51.905 INFO  [CHM] Incoming seq no is better than mine for node 2. Merging data
000027b8.00001428::2015/08/25-11:33:51.905 INFO  [CHM] My weights have changed: <vector len='65'>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>111</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 INFO  </vector>
000027b8.00001428::2015/08/25-11:33:51.905 INFO  .
000027b8.00001428::2015/08/25-11:33:51.905 INFO  [CHM] Sending route weight vector for nodes (1 2) to nodes (2)
000027b8.00002ddc::2015/08/25-11:33:52.905 INFO  [CHM] Sending route weight vector for nodes (1 2) to nodes (2)
000027b8.00002ddc::2015/08/25-11:33:54.046 DBG   [NETFTAPI] received NsiParameterNotification for 10.0.160.51 (IpDadStatePreferred)
000027b8.00002ddc::2015/08/25-11:33:54.061 DBG   [NETFTAPI] Signaled NetftLocalConnect event for 10.0.160.51
000027b8.00001f00::2015/08/25-11:33:54.061 INFO  [IM] got event: Local endpoint 10.0.160.51:~0~ connected
000027b8.00001f00::2015/08/25-11:33:54.061 INFO  [IM] Adding information for route Route from local 10.0.160.51:~3343~ to remote 10.0.160.52:~3343~, status: false, attributes: 0
000027b8.00001f00::2015/08/25-11:33:54.061 INFO  [IM] Sending connectivity report to leader (node 1): <class mscs::InterfaceReport>
000027b8.00001f00::2015/08/25-11:33:54.061 INFO    <fromInterface>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</fromInterface>
000027b8.00001f00::2015/08/25-11:33:54.061 INFO    <upInterfaces><vector len='1'>
000027b8.00001f00::2015/08/25-11:33:54.061 INFO      <item>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</item>
000027b8.00001f00::2015/08/25-11:33:54.061 INFO  </vector>
000027b8.00001f00::2015/08/25-11:33:54.061 INFO  </upInterfaces>
000027b8.00001f00::2015/08/25-11:33:54.061 INFO    <downInterfaces><vector len='1'>
000027b8.00001f00::2015/08/25-11:33:54.061 INFO      <item>5f354f39-8e3b-4ad1-bb13-96e2e1d75d55</item>
000027b8.00001f00::2015/08/25-11:33:54.061 INFO  </vector>
000027b8.00001f00::2015/08/25-11:33:54.061 INFO  </downInterfaces>
000027b8.00001f00::2015/08/25-11:33:54.061 INFO    <upRoutesType><vector len='0'>
000027b8.00001f00::2015/08/25-11:33:54.061 INFO  </vector>
000027b8.00001f00::2015/08/25-11:33:54.061 INFO  </upRoutesType>
000027b8.00001f00::2015/08/25-11:33:54.061 INFO    <downRoutesType><vector len='1'>
000027b8.00001f00::2015/08/25-11:33:54.061 INFO      <item>1</item>
000027b8.00001f00::2015/08/25-11:33:54.061 INFO  </vector>
000027b8.00001f00::2015/08/25-11:33:54.061 INFO  </downRoutesType>
000027b8.00001f00::2015/08/25-11:33:54.061 INFO    <viewId>201</viewId>
000027b8.00001f00::2015/08/25-11:33:54.061 INFO    <localDisconnect>false</localDisconnect>
000027b8.00001f00::2015/08/25-11:33:54.061 INFO  </class mscs::InterfaceReport>
000027b8.00002438::2015/08/25-11:33:54.061 INFO  [IM] Leader got report from 1
000027b8.00002438::2015/08/25-11:33:54.061 INFO  [IM - Heartbeat Network] 1 reports in state calculator queue
000027b8.00002410::2015/08/25-11:33:54.061 INFO  [IM - Heartbeat Network] State calculator got new report from 5ced600f-83b3-4fde-aa4e-f3d2e4f6e584
000027b8.00002410::2015/08/25-11:33:54.061 INFO  [IM - Heartbeat Network] 0 reports in state calculator queue
000027b8.00001428::2015/08/25-11:33:54.061 DBG   [NETFTAPI] received NsiAddInstance for fe80::5efe:10.0.160.51
000027b8.00002ddc::2015/08/25-11:33:54.077 DBG   [NETFTAPI] received NsiParameterNotification for fe80::5efe:10.0.160.51 (IpDadStateDeprecated)
000027b8.000021b0::2015/08/25-11:33:54.827 DBG   [NETFTAPI] Signaled NetftRemoteReachable event, local address 10.0.160.51:3343 remote address 10.0.160.52:3343
000027b8.00001f00::2015/08/25-11:33:54.827 INFO  [TM] got event: Remote endpoint 10.0.160.52:~3343~ reachable from 10.0.160.51:~3343~
000027b8.00001f00::2015/08/25-11:33:54.827 INFO  [IM] got event: Remote endpoint 10.0.160.52:~3343~ reachable from 10.0.160.51:~3343~
000027b8.00001f00::2015/08/25-11:33:54.827 INFO  [IM] Marking Route from 10.0.160.51:~3343~ to 10.0.160.52:~3343~ as up
000027b8.00002ddc::2015/08/25-11:33:54.827 INFO  [DCM] HandleNetftRemoteRouteChange
000027b8.000013c4::2015/08/25-11:33:54.827 INFO  [DCM] HandleRequest: dcm/netftRouteChange
000027b8.00001f00::2015/08/25-11:33:54.827 INFO  [IM] Adding information for route Route from local 10.0.160.51:~3343~ to remote 10.0.160.52:~3343~, status: true, attributes: 0
000027b8.00001f00::2015/08/25-11:33:54.827 INFO  [IM] Sending connectivity report to leader (node 1): <class mscs::InterfaceReport>
000027b8.00001f00::2015/08/25-11:33:54.827 INFO    <fromInterface>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</fromInterface>
000027b8.000013c4::2015/08/25-11:33:54.827 INFO  [DCM] Skipping client access network d7dcde53-bf75-427e-8083-c2f4166be39c for multichannel
000027b8.00001f00::2015/08/25-11:33:54.827 INFO    <upInterfaces><vector len='2'>
000027b8.00001f00::2015/08/25-11:33:54.827 INFO      <item>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</item>
000027b8.00001f00::2015/08/25-11:33:54.827 INFO      <item>5f354f39-8e3b-4ad1-bb13-96e2e1d75d55</item>
000027b8.00001f00::2015/08/25-11:33:54.827 INFO  </vector>
000027b8.00001f00::2015/08/25-11:33:54.827 INFO  </upInterfaces>
000027b8.00001f00::2015/08/25-11:33:54.827 INFO    <downInterfaces><vector len='0'>
000027b8.00001f00::2015/08/25-11:33:54.827 INFO  </vector>
000027b8.00001f00::2015/08/25-11:33:54.827 INFO  </downInterfaces>
000027b8.00001f00::2015/08/25-11:33:54.827 INFO    <upRoutesType><vector len='1'>
000027b8.00001f00::2015/08/25-11:33:54.827 INFO      <item>1</item>
000027b8.00001f00::2015/08/25-11:33:54.827 INFO  </vector>
000027b8.00001f00::2015/08/25-11:33:54.827 INFO  </upRoutesType>
000027b8.00001f00::2015/08/25-11:33:54.827 INFO    <downRoutesType><vector len='0'>
000027b8.00001f00::2015/08/25-11:33:54.827 INFO  </vector>
000027b8.00001f00::2015/08/25-11:33:54.827 INFO  </downRoutesType>
000027b8.00001f00::2015/08/25-11:33:54.827 INFO    <viewId>201</viewId>
000027b8.00001f00::2015/08/25-11:33:54.827 INFO    <localDisconnect>false</localDisconnect>
000027b8.00001f00::2015/08/25-11:33:54.827 INFO  </class mscs::InterfaceReport>
000027b8.00002438::2015/08/25-11:33:54.827 INFO  [IM] Leader got report from 1
000027b8.00002438::2015/08/25-11:33:54.827 INFO  [IM - Heartbeat Network] 1 reports in state calculator queue
000027b8.00002410::2015/08/25-11:33:54.827 INFO  [IM - Heartbeat Network] State calculator got new report from 5ced600f-83b3-4fde-aa4e-f3d2e4f6e584
000027b8.00002410::2015/08/25-11:33:54.827 INFO  [IM - Heartbeat Network] 0 reports in state calculator queue
000027b8.00002448::2015/08/25-11:33:54.827 INFO  [IM] Leader got report from 2
000027b8.00002448::2015/08/25-11:33:54.827 INFO  [IM - Heartbeat Network] 1 reports in state calculator queue
000027b8.00002410::2015/08/25-11:33:54.827 INFO  [IM - Heartbeat Network] State calculator got new report from 5f354f39-8e3b-4ad1-bb13-96e2e1d75d55
000027b8.00002410::2015/08/25-11:33:54.827 INFO  [IM - Heartbeat Network] 0 reports in state calculator queue
000027b8.000013c4::2015/08/25-11:33:54.827 INFO  [DCM] Registering name fe80::8cce:1c3c:9d53:51cd for multichannel support returned 0
000027b8.000013c4::2015/08/25-11:33:54.827 INFO  [DCM] Sending local node id to node Id 2
000027b8.00002410::2015/08/25-11:33:54.827 INFO  (allSplitGroups, splitGroups) = [IM - Heartbeat Network] Two splits for group (0 1)
000027b8.00002410::2015/08/25-11:33:54.827 INFO  [IM - Heartbeat Network] Calculating equal interface state for groups (0) and (1)
000027b8.00002410::2015/08/25-11:33:54.827 INFO  [IM - Heartbeat Network] Calculated interface state result: <class mscs::InterfaceResult>
000027b8.00002410::2015/08/25-11:33:54.827 INFO    <up><vector len='0'>
000027b8.00002410::2015/08/25-11:33:54.827 INFO  </vector>
000027b8.00002410::2015/08/25-11:33:54.827 INFO  </up>
000027b8.00002410::2015/08/25-11:33:54.827 INFO    <down><vector len='0'>
000027b8.00002410::2015/08/25-11:33:54.827 INFO  </vector>
000027b8.00002410::2015/08/25-11:33:54.827 INFO  </down>
000027b8.00002410::2015/08/25-11:33:54.827 INFO    <unreachable><vector len='2'>
000027b8.00002410::2015/08/25-11:33:54.827 INFO      <item>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</item>
000027b8.00002410::2015/08/25-11:33:54.827 INFO      <item>5f354f39-8e3b-4ad1-bb13-96e2e1d75d55</item>
000027b8.00002410::2015/08/25-11:33:54.827 INFO  </vector>
000027b8.00002410::2015/08/25-11:33:54.827 INFO  </unreachable>
000027b8.00002410::2015/08/25-11:33:54.827 INFO  </class mscs::InterfaceResult>
000027b8.00002410::2015/08/25-11:33:54.827 INFO  [IM - Heartbeat Network] Issuing state change update with result <class mscs::InterfaceResult>
000027b8.00002410::2015/08/25-11:33:54.827 INFO    <up><vector len='0'>
000027b8.00002410::2015/08/25-11:33:54.827 INFO  </vector>
000027b8.00002410::2015/08/25-11:33:54.827 INFO  </up>
000027b8.00002410::2015/08/25-11:33:54.827 INFO    <down><vector len='0'>
000027b8.00002410::2015/08/25-11:33:54.827 INFO  </vector>
000027b8.00002410::2015/08/25-11:33:54.827 INFO  </down>
000027b8.00002410::2015/08/25-11:33:54.827 INFO    <unreachable><vector len='2'>
000027b8.00002410::2015/08/25-11:33:54.827 INFO      <item>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</item>
000027b8.00002410::2015/08/25-11:33:54.827 INFO      <item>5f354f39-8e3b-4ad1-bb13-96e2e1d75d55</item>
000027b8.00002410::2015/08/25-11:33:54.827 INFO  </vector>
000027b8.00002410::2015/08/25-11:33:54.827 INFO  </unreachable>
000027b8.00002410::2015/08/25-11:33:54.827 INFO  </class mscs::InterfaceResult>
000027b8.00002410::2015/08/25-11:33:54.827 INFO  [GEM] Node 1: Sending 1 messages as a batched GEM message
000027b8.00002410::2015/08/25-11:33:54.827 INFO  [IM - Heartbeat Network] State calculation complete without issuing pings
000027b8.00002410::2015/08/25-11:33:54.827 INFO  [IM - Heartbeat Network] Resetting interface state calculation state
000027b8.000027bc::2015/08/25-11:33:54.827 INFO  [IM] Changing the state of adapters according to result: <class mscs::InterfaceResult>
000027b8.000027bc::2015/08/25-11:33:54.827 INFO    <up><vector len='0'>
000027b8.000027bc::2015/08/25-11:33:54.827 INFO  </vector>
000027b8.000027bc::2015/08/25-11:33:54.827 INFO  </up>
000027b8.000027bc::2015/08/25-11:33:54.827 INFO    <down><vector len='0'>
000027b8.000027bc::2015/08/25-11:33:54.827 INFO  </vector>
000027b8.000027bc::2015/08/25-11:33:54.827 INFO  </down>
000027b8.000027bc::2015/08/25-11:33:54.827 INFO    <unreachable><vector len='2'>
000027b8.000027bc::2015/08/25-11:33:54.827 INFO      <item>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</item>
000027b8.000027bc::2015/08/25-11:33:54.827 INFO      <item>5f354f39-8e3b-4ad1-bb13-96e2e1d75d55</item>
000027b8.000027bc::2015/08/25-11:33:54.827 INFO  </vector>
000027b8.000027bc::2015/08/25-11:33:54.827 INFO  </unreachable>
000027b8.000027bc::2015/08/25-11:33:54.827 INFO  </class mscs::InterfaceResult>
000027b8.0000212c::2015/08/25-11:33:54.827 INFO  [DCM] HandleInterfaceChange
000027b8.000013c4::2015/08/25-11:33:54.827 INFO  [DCM] HandleRequest: dcm/connectivityCheck
000027b8.000013c4::2015/08/25-11:33:54.827 INFO  [DCM] Skipping client access network d7dcde53-bf75-427e-8083-c2f4166be39c for multichannel
000027b8.00002574::2015/08/25-11:33:54.827 INFO  [NM] Changing network state for network Heartbeat Network to 2
000027b8.00002574::2015/08/25-11:33:54.827 INFO  [GEM] Node 1: Sending 1 messages as a batched GEM message
000027b8.00002574::2015/08/25-11:33:54.827 INFO  [NM] Received state change update for network Heartbeat Network (id 6232c658-0077-4a50-978e-6415c4de564e) to 2
000027b8.00001428::2015/08/25-11:33:54.905 INFO  [CHM] Sending route weight vector for nodes (1 2) to nodes (2)
000027b8.00002448::2015/08/25-11:33:55.140 INFO  [IM] Leader got report from 2
000027b8.00002448::2015/08/25-11:33:55.140 INFO  [IM - Heartbeat Network] 1 reports in state calculator queue
000027b8.00002410::2015/08/25-11:33:55.140 INFO  [IM - Heartbeat Network] State calculator got new report from 5f354f39-8e3b-4ad1-bb13-96e2e1d75d55
000027b8.00002410::2015/08/25-11:33:55.140 INFO  [IM - Heartbeat Network] 0 reports in state calculator queue
000027b8.000013c4::2015/08/25-11:33:55.140 INFO  [DCM] HandleRequest: dcm/queryServerEndpoints
000027b8.000013c4::2015/08/25-11:33:55.140 WARN  [DCM] RDR handle for target node id 2 is not yet populated
000027b8.000013c4::2015/08/25-11:33:55.140 WARN  [DCM] RDR handle to node 2 is not available. Skipping server refresh
000027b8.00001428::2015/08/25-11:33:58.905 INFO  [CHM] Sending route weight vector for nodes (1 2) to nodes (2)
000027b8.00001428::2015/08/25-11:34:01.140 INFO  [IM - Heartbeat Network] Timer fired. Sending off request to get reports about missing interfaces <vector len='2'>
000027b8.00001428::2015/08/25-11:34:01.140 INFO      <item></item>

Any help will be greatly appreciated. I also have another cluster using the same heartbeat network and I do not get these errors in the other cluster.

↧
Search
Viewing all articles
Browse latest Browse all 5654

Trending Articles

Practice Sheet of Right form of verbs for HSC Students

September 22, 2019, 11:40 pm

Download: FK ft Shenky – Nakuyewa ”Prod by: Shenky”

February 16, 2017, 4:24 pm

How to win at Markstrat (Markstrat Tips and Tricks) – Vodites

January 5, 2014, 10:34 pm

Ominde Commission Report and Recommendations – Ominde Report of 1964

March 16, 2015, 5:14 am

Bureau of Internal Revenue: Regional Offices (Directory)

January 9, 2014, 11:06 pm

GO 53 on Enhancement of Ex-gratia upto 5 Lakhs Toddy Tappers in Telangana

March 26, 2017, 11:23 pm

Cakewalk CA-2A Leveling Amplifier v2.0.1.97 WiN, v2.0.1.96 OSX Incl Keygen

October 17, 2016, 7:20 am

Mp3 Download: Mdu - Kunjenjenjena

December 7, 2017, 8:16 am

How the kill the job , when DTP request running for long hours.

July 26, 2013, 2:41 am

Microsoft Intune から展開しているアプリのアップデートについて

October 17, 2016, 4:11 am

18-year-old girl was beaten for half an hour by two Northampton men in 'an...

September 1, 2017, 10:00 pm

Car crash in Dunton Bassett leaves driver in critical condition

October 7, 2014, 7:51 am

Macky 2, Two Others In Road Accident

March 29, 2015, 5:34 am

Application log 00000000000000089514: Could not convert queue DLVST90CLNT

May 14, 2015, 11:27 pm

Detroit mafia: D’Anna Brothers agree to plea deal

April 21, 2016, 6:56 am

Delivery block field greyed out using VA02

January 26, 2016, 2:52 pm

Muloraki Au

June 22, 2016, 1:44 am

【個人撮影】スマホのプライベート映像♪「中に出さないで///」カラオケ屋での生ハメ撮りが流出ｗ【リベンジポルノ】＠PornHub

October 12, 2017, 2:23 pm

BREAKING NEWS: Diamond Platnumz Is Reported Dead After Ghastly Car Accident

February 9, 2018, 4:56 am

FIAT 500 B0111 B0112

July 5, 2018, 10:31 am
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>