I've got a two node 2008 R2 failover cluster. I have a single service being managed by it that I configured just as a generic service. The failover works perfectly when the service is stopped, or when one of the machines goes down, and the immediate failback I have configured works perfectly in both scenarios as well.
However, there's an issue when I take the networking down on the preferred owner of the service. As far as I can tell (this is the first time I've tried failover clustering, so I'm learning), when I take the networking down, the cluster service shuts down, and in turn shuts down the service I've told it to manage. At this point, when the services aren't running, the service fails over to the secondary as intended. The problem shows up when I turn the networking back on. The service tries and fails to start on the primary (as many times as I've configured it to try), and then eventually gives up and goes back to the secondary.
The reason for this, examining logs for the service, is that the required port is already in use. I checked some more, and sure enough, when I take the networking offline the service gets shut down, but the executable is still running. This is repeatable every time. When I just stop the service, though, the executables go away. So it's something to do specifically with how the managed service gets shut down *when it's shut down due to the cluster service stopping*. For some reason it's not cleaning up that associated executable.
Any ideas as to why this is happening and how to fix/work around it would be extremely welcome. Thank you!