Hi
I have a problem with failover clustering and iSCSI target in Windows Server 2016.
I often demo the configuration of Failover Clustering for students and customers. And for that I have built a small lab consisting of a Domain Controller and three servers functioning as my cluster nodes.
All Servers run Windows Server 2016 (1607) and have been fully updated with the latest patches. All servers are virtual machines running in Hyper-V on Windows 10 v1709/1803.
The Domain Controller functions as Domain Controller and I have also added the iSCSI role service on that as well. And yes, I know that is not the way to do but this is only a test/demo environment. I have created two iSCSI disk, one for the Witness (1 GB) and one for Shared Storage (100 GB).
My three nodes also run Windows Server 2016 (1607) and is configured with two network cards, one for management (LAN) and one for cluster traffic. I have installed the Failover Clustering feature on these servers and are using the built-in iSCSI initiator to connect to the shared storage published by my iSCSI target running on the Domain Controller. And this is where I run into the first problem.
In order to give the nodes (servers) access to the storage they need to be added to theInitiators list from within Server Manger ->File and Storage Services -> iSCSI. If I choose theQuery initiator computer for ID options and then click Browse (in order to browse for the names of the nodes) Server Manager crashes and I get a window stating that Server Manager has stopped working. If I look in the Event Viewer I see the following events:
Log Name: Application
Source: .NET Runtime
Date: 04-06-2018 17:36:52
Event ID: 1026
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: cph-dc-01.ad.petzfeed.com
Description:
Application: ServerManager.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Application: ServerManager.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.ArgumentException
at Microsoft.FileServer.Management.Plugin.Dialogues.BrowseDSObjectsNativeMethods+IDsObjectPicker.Initialize(DSOP_INIT_INFO ByRef)
at Microsoft.FileServer.Management.Plugin.Dialogues.BrowseDSObjectsDialog.ShowDialog(System.Windows.Forms.IWin32Window, PickerTypes, Boolean, StartScope, ProviderTypes, System.String, System.Security.SecureString)
at Microsoft.FileServer.Management.Plugin.Services.DialogService.ShowPickDialog(PickerTypes, Microsoft.FileServer.Management.Framework.ComputerName, StartScope, ReturnSourceTypes, System.String ByRef)
at Microsoft.FileServer.Management.Plugin.Services.DialogService.ShowPickComputerDialog(Microsoft.FileServer.Management.Framework.ComputerName, Microsoft.FileServer.Management.Framework.ComputerName ByRef)
at Microsoft.FileServer.Management.Plugin.Dialogues.AddInitiatorIdSectionDescriptor+<>c__DisplayClass41_0.<.ctor>b__0()
at MS.Internal.Commands.CommandHelpers.CriticalExecuteCommandSource(System.Windows.Input.ICommandSource, Boolean)
at System.Windows.Controls.Primitives.ButtonBase.OnClick()
at System.Windows.Controls.Button.OnClick()
at System.Windows.Controls.Primitives.ButtonBase.OnMouseLeftButtonUp(System.Windows.Input.MouseButtonEventArgs)
at System.Windows.RoutedEventArgs.InvokeHandler(System.Delegate, System.Object)
at System.Windows.RoutedEventHandlerInfo.InvokeHandler(System.Object, System.Windows.RoutedEventArgs)
at System.Windows.EventRoute.InvokeHandlersImpl(System.Object, System.Windows.RoutedEventArgs, Boolean)
at System.Windows.UIElement.ReRaiseEventAs(System.Windows.DependencyObject, System.Windows.RoutedEventArgs, System.Windows.RoutedEvent)
at System.Windows.UIElement.OnMouseUpThunk(System.Object, System.Windows.Input.MouseButtonEventArgs)
at System.Windows.RoutedEventArgs.InvokeHandler(System.Delegate, System.Object)
at System.Windows.RoutedEventHandlerInfo.InvokeHandler(System.Object, System.Windows.RoutedEventArgs)
at System.Windows.EventRoute.InvokeHandlersImpl(System.Object, System.Windows.RoutedEventArgs, Boolean)
at System.Windows.UIElement.RaiseEventImpl(System.Windows.DependencyObject, System.Windows.RoutedEventArgs)
at System.Windows.UIElement.RaiseTrustedEvent(System.Windows.RoutedEventArgs)
at System.Windows.Input.InputManager.ProcessStagingArea()
at System.Windows.Input.InputManager.ProcessInput(System.Windows.Input.InputEventArgs)
at System.Windows.Input.InputProviderSite.ReportInput(System.Windows.Input.InputReport)
at System.Windows.Interop.HwndMouseInputProvider.ReportInput(IntPtr, System.Windows.Input.InputMode, Int32, System.Windows.Input.RawMouseActions, Int32, Int32, Int32)
at System.Windows.Interop.HwndMouseInputProvider.FilterMessage(IntPtr, MS.Internal.Interop.WindowMessage, IntPtr, IntPtr, Boolean ByRef)
at System.Windows.Interop.HwndSource.InputFilterMessage(IntPtr, Int32, IntPtr, IntPtr, Boolean ByRef)
at MS.Win32.HwndWrapper.WndProc(IntPtr, Int32, IntPtr, IntPtr, Boolean ByRef)
at MS.Win32.HwndSubclass.DispatcherCallbackOperation(System.Object)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(System.Delegate, System.Object, Int32)
at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(System.Object, System.Delegate, System.Object, Int32, System.Delegate)
at System.Windows.Threading.Dispatcher.LegacyInvokeImpl(System.Windows.Threading.DispatcherPriority, System.TimeSpan, System.Delegate, System.Object, Int32)
at MS.Win32.HwndSubclass.SubclassWndProc(IntPtr, Int32, IntPtr, IntPtr)
at MS.Win32.UnsafeNativeMethods.DispatchMessage(System.Windows.Interop.MSG ByRef)
at System.Windows.Threading.Dispatcher.PushFrameImpl(System.Windows.Threading.DispatcherFrame)
at System.Windows.Window.ShowHelper(System.Object)
at System.Windows.Window.ShowDialog()
at Microsoft.FileServer.Management.Plugin.Services.DialogService.ShowAddInitiatorIdDialog(Microsoft.FileServer.Management.Framework.ComputerName, Microsoft.FileServer.Management.Plugin.Model.InitiatorId ByRef)
at Microsoft.FileServer.Management.Plugin.PropertyPages.IscsiTargetInitiatorsPropertySectionDescriptor.<.ctor>b__4_0()
at MS.Internal.Commands.CommandHelpers.CriticalExecuteCommandSource(System.Windows.Input.ICommandSource, Boolean)
at System.Windows.Controls.Primitives.ButtonBase.OnClick()
at System.Windows.Controls.Button.OnClick()
at System.Windows.Controls.Primitives.ButtonBase.OnMouseLeftButtonUp(System.Windows.Input.MouseButtonEventArgs)
at System.Windows.RoutedEventArgs.InvokeHandler(System.Delegate, System.Object)
at System.Windows.RoutedEventHandlerInfo.InvokeHandler(System.Object, System.Windows.RoutedEventArgs)
at System.Windows.EventRoute.InvokeHandlersImpl(System.Object, System.Windows.RoutedEventArgs, Boolean)
at System.Windows.UIElement.ReRaiseEventAs(System.Windows.DependencyObject, System.Windows.RoutedEventArgs, System.Windows.RoutedEvent)
at System.Windows.UIElement.OnMouseUpThunk(System.Object, System.Windows.Input.MouseButtonEventArgs)
at System.Windows.RoutedEventArgs.InvokeHandler(System.Delegate, System.Object)
at System.Windows.RoutedEventHandlerInfo.InvokeHandler(System.Object, System.Windows.RoutedEventArgs)
at System.Windows.EventRoute.InvokeHandlersImpl(System.Object, System.Windows.RoutedEventArgs, Boolean)
at System.Windows.UIElement.RaiseEventImpl(System.Windows.DependencyObject, System.Windows.RoutedEventArgs)
at System.Windows.UIElement.RaiseTrustedEvent(System.Windows.RoutedEventArgs)
at System.Windows.Input.InputManager.ProcessStagingArea()
at System.Windows.Input.InputManager.ProcessInput(System.Windows.Input.InputEventArgs)
at System.Windows.Input.InputProviderSite.ReportInput(System.Windows.Input.InputReport)
at System.Windows.Interop.HwndMouseInputProvider.ReportInput(IntPtr, System.Windows.Input.InputMode, Int32, System.Windows.Input.RawMouseActions, Int32, Int32, Int32)
at System.Windows.Interop.HwndMouseInputProvider.FilterMessage(IntPtr, MS.Internal.Interop.WindowMessage, IntPtr, IntPtr, Boolean ByRef)
at System.Windows.Interop.HwndSource.InputFilterMessage(IntPtr, Int32, IntPtr, IntPtr, Boolean ByRef)
at MS.Win32.HwndWrapper.WndProc(IntPtr, Int32, IntPtr, IntPtr, Boolean ByRef)
at MS.Win32.HwndSubclass.DispatcherCallbackOperation(System.Object)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(System.Delegate, System.Object, Int32)
at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(System.Object, System.Delegate, System.Object, Int32, System.Delegate)
at System.Windows.Threading.Dispatcher.LegacyInvokeImpl(System.Windows.Threading.DispatcherPriority, System.TimeSpan, System.Delegate, System.Object, Int32)
at MS.Win32.HwndSubclass.SubclassWndProc(IntPtr, Int32, IntPtr, IntPtr)
at MS.Win32.UnsafeNativeMethods.DispatchMessage(System.Windows.Interop.MSG ByRef)
at System.Windows.Threading.Dispatcher.PushFrameImpl(System.Windows.Threading.DispatcherFrame)
at System.Windows.Application.RunDispatcher(System.Object)
at System.Windows.Application.RunInternal(System.Windows.Window)
at Microsoft.Windows.ServerManager.SingleInstanceAppLauncher.StartApplication(Microsoft.Windows.ServerManager.Common.ArgumentsProcessor)
at Microsoft.Windows.ServerManager.MainApplication.Main(System.String[])
And this error as well:
Faulting application name: ServerManager.exe, version: 10.0.14393.1358, time stamp: 0x593272e2
Faulting module name: KERNELBASE.dll, version: 10.0.14393.1532, time stamp: 0x5965ac8c
Exception code: 0xe0434352
Fault offset: 0x0000000000033c58
Faulting process id: 0xf2c
Faulting application start time: 0x01d3fc19440b98b8
Faulting application path: C:\Windows\system32\ServerManager.exe
Faulting module path: C:\Windows\System32\KERNELBASE.dll
Report Id: a9a4656c-453c-4ef6-8d83-165ee783cbfd
Faulting package full name:
Faulting package-relative application ID:
If I choose theEnter a value for the selected type and select DNS and browse for the machine name the same thing happens – Server Manager crashes and the same events is reported in Event Viewer.
However, I can work around this issue by using the IP addresses of my three nodes instead and add them to the Initiators list. When all that is done my three nodes have no problem connecting to the iSCSI storage.
The real problem comes when I try to run the Validate a Configuration Wizard (Cluster wizard) from within the Failover Cluster Manager. When I clickBrowse (in order to search for the servers, I want to validate) the wizard closes without any visible errors. In the Event Viewer the following error is reported:
Log Name: Microsoft-Windows-FailoverClustering-Manager/Admin
Source: Microsoft-Windows-FailoverClustering-Manager
Date: 04-06-2018 18:12:52
Event ID: 4709
Task Category: MMC Snapin
Level: Error
Keywords:
User: AD\mbucadmin
Computer: cph-hv-01.ad.petzfeed.com
Description:
Failover Cluster Manager encountered a fatal error.
System.ApplicationException: Unable to browse for computer objects. ---> System.ArgumentException: Value does not fall within the expected range.
at MS.Internal.ServerClusters.NativeMethods.IDsObjectPicker.Initialize(DSOP_INIT_INFO& pInitInfo)
at MS.Internal.ServerClusters.BrowseDSObjectsDialog.ShowDialog(IWin32Window owner, PickerTypes pickerType, Boolean multipleSelect)
at MS.Internal.ServerClusters.BrowseDSObjectsDialog.ShowPickComputerDialog(IWin32Window owner, Boolean multipleSelect)
--- End of inner exception stack trace ---
at MS.Internal.ServerClusters.BrowseDSObjectsDialog.ShowPickComputerDialog(IWin32Window owner, Boolean multipleSelect)
at MS.Internal.ServerClusters.Wizards.SelectItemsPage.OnBrowseClicked(Object sender, EventArgs e)
at System.Windows.Forms.Control.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ButtonBase.WndProc(Message& m)
at System.Windows.Forms.Button.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
System.ArgumentException: Value does not fall within the expected range.
at MS.Internal.ServerClusters.NativeMethods.IDsObjectPicker.Initialize(DSOP_INIT_INFO& pInitInfo)
at MS.Internal.ServerClusters.BrowseDSObjectsDialog.ShowDialog(IWin32Window owner, PickerTypes pickerType, Boolean multipleSelect)
at MS.Internal.ServerClusters.BrowseDSObjectsDialog.ShowPickComputerDialog(IWin32Window owner, Boolean multipleSelect)
Again, I can work around this by typing the name of my nodes and adding them one at time by clickingAdd. Then the validation begins and the List Disk to Be Validated test fails every time with the following error:
* Failed while verifying removal of any Persistent Reservation on physical disk {24dea968-82e9-4dbf-aacd-a0fe00236632} at node cph-hv-01.ad.petzfeed.com.
* Failed while verifying removal of any Persistent Reservation on physical disk {24dea968-82e9-4dbf-aacd-a0fe00236632} at node cph-hv-01.ad.petzfeed.com.
If I just ignore the error and create the cluster anyway, it doesn´t function properly and complains about access to the storage.
Here is what I have tried:
- Recreating the virtual machines using different installation media but that makes no difference.
- Tried using three different Windows 10 machines when running the virtual machines but that doesn´t help either.
- Using two different Synology boxes as iSCSI target but it is the same.
- Tried using both Gen 1 and Gen 2 virtual machines with or without Secure boot enabled.
But the real funny part is that if I use the Windows Server 2016 RTM media (1607) and choose not to patch (update) the nodes everything works as expected. If I apply the updates after the cluster is up and running it seems to work okay. If I update the servers before I setup the cluster I run in to the issues described above. If I setup the same environment using Windows Server 2012 R2 it just works.
Are anyone out there able to reproduce what I am seeing or maybe someone has seen this problem before.