Windows Team

Just another bunch of Windows system administrators

Unable to Hyper-V Live Migrate in Windows Server 2012

📥  hyper-v

Saw some interesting Windows Server 2012 Clustering behavior recently.
We noticed that we were unable to perform any live migrations on one of our clusters.
Looking in Cluster Events we saw a lot errors - Event Id 1196 coming up regularly, around every 10-15 minutes or so. We also saw errors for Event Id 1206 when restarting the Cluster Name Resource.
Event Id 1196 -
Cluster network name resource 'Cluster Name' failed registration of one or more associated DNS name(s) for the following reason:
The handle is invalid.
Event Id 1206 -
The computer object associated with the cluster network name resource 'Cluster Name' could not be updated in domain 'campus.bath.ac.uk'. The error code was 'Password change'. The cluster identity 'xxxxxxx

may lack permissions required to update the object. Please work with your domain administrator to ensure that the cluster identity can update computer objects in the domain.

The cluster logs also showed some interesting errors -
00000ef0.00002cb0::2013/06/17-15:13:28.905 WARN  [RES] Network Name: [NNLIB] LogonUserEx fails for user xxxxxxx$: 1326 (useSecondaryPassword: 0)
00000ef0.00002cb0::2013/06/17-15:13:28.961 WARN  [RES] Network Name: [NNLIB] LogonUserEx fails for user  xxxxxxx$: 1326 (useSecondaryPassword: 1)
00000ef0.00002cb0::2013/06/17-15:13:28.961 INFO  [RES] Network Name: [NNLIB] Logon failed for user megadrive$ (Error 1326), DC \xxxxxx.campus.bath.ac.uk, domain campus.bath.ac.uk
"Binging" around - really, do people Bing things? - "Googling" around (Do you think the most Binged thing is "How do I set Google as my default search engine in IE?") we came across of lots interesting tails about how people had rebuilt thier cluster, or reverted to Windows 2008 R2 because the problems weren't seen there.
And then we stumbled across a blog by the Windows Server Core Team - http://blogs.technet.com/b/askcore/archive/2012/09/25/cno-blog-series-increasing-awareness-around-the-cluster-name-object-cno.aspx about how sysadmins need more 'awareness' about Cluster Name Object (CNO). When a Cluster is created the computer object for the Cluster is created in the Computers container. If it is moved to another OU then "the non-default location may not have the rights it needs for other cluster operations" . We had moved ours to a different OU. We also saw a number of other articles that were similar to our problem but not the same - http://www.andrewparisio.com/2012/12/windows-failover-cluster-live-migration.html
http://social.technet.microsoft.com/Forums/windowsserver/en-US/2ad0afaf-8d86-4f16-b748-49bf9ac447a3/ws2012-cluster-network-dns-issues
This was our fix -
1 - Move CNO back to the Computers Container
2- Give the Cluster Node Computer Accounts Change Password permission on the CNO
3 - Take the Cluster Name Resource offline
4 - Repair Cluster Name Resource
5 - Bring Cluster Name Resource back online
Job done, can now Live Migrate
Hope this helps anyone in a similar situation

Saw some interesting behavior recently in Windows Server 2012 Clustering.

We noticed that we were unable to perform any live migrations on one of our clusters.

Looking in Cluster Events we saw a lot errors - Event Id 1196 coming up regularly, around every 10-15 minutes or so. We also saw errors for Event Id 1206 when restarting the Cluster Name Resource.

Event Id 1196 -

Cluster network name resource 'Cluster Name' failed registration of one or more associated DNS name(s) for the following reason:

The handle is invalid.

Event Id 1206 -

The computer object associated with the cluster network name resource 'Cluster Name' could not be updated in domain 'xxxxx.xxxx'. The error code was 'Password change'. The cluster identity 'xxxxxxx$' may lack permissions required to update the object. Please work with your domain administrator to ensure that the cluster identity can update computer objects in the domain.

The cluster logs also showed some interesting errors -

00000ef0.00002cb0::2013/06/17-15:13:28.905 WARN  [RES] Network Name: [NNLIB] LogonUserEx fails for user xxxxxxx$: 1326 (useSecondaryPassword: 0)

00000ef0.00002cb0::2013/06/17-15:13:28.961 WARN  [RES] Network Name: [NNLIB] LogonUserEx fails for user  xxxxxxx$: 1326 (useSecondaryPassword: 1)

00000ef0.00002cb0::2013/06/17-15:13:28.961 INFO  [RES] Network Name: [NNLIB] Logon failed for user megadrive$ (Error 1326), DC \xxxxxx.xxxx.xxxx, domain campus.bath.ac.uk

"Binging" around - really, do people Bing things? - "Googling" around (Do you think the most Binged thing is "How do I set Google as my default search engine in IE?") we came across of lots interesting tails about how people had rebuilt thier cluster, or reverted to Windows 2008 R2 because the problems weren't seen there.

And then we stumbled across a blog by the Windows Server Core Team - http://blogs.technet.com/b/askcore/archive/2012/09/25/cno-blog-series-increasing-awareness-around-the-cluster-name-object-cno.aspx about how sysadmins need more 'awareness' about Cluster Name Object (CNO). When a Cluster is created the computer object for the Cluster is created in the Computers container. If it is moved to another OU then "the non-default location may not have the rights it needs for other cluster operations" . We had moved ours to a different OU. We also saw a number of other articles that were similar to our problem but not the same - http://www.andrewparisio.com/2012/12/windows-failover-cluster-live-migration.html

http://social.technet.microsoft.com/Forums/windowsserver/en-US/2ad0afaf-8d86-4f16-b748-49bf9ac447a3/ws2012-cluster-network-dns-issues

This was our fix -

1 - Move CNO back to the Computers Container

2- Give the Cluster Node Computer Accounts Change Password permission on the CNO

3 - Take the Cluster Name Resource offline

4 - Repair Cluster Name Resource

5 - Bring Cluster Name Resource back online

Job done, can now Live Migrate

Hope this helps anyone in a similar situation

A year is virtually a long time

  , , , ,

📥  hyper-v

It seems a long time ago since Mark and I turned up at a training company in London to find out that we were the only attendees on an Implementing and Managing Windows Server 2008 Hyper-V course.

In fact it's only been a year. So how far have we come since we came back to office, bright eyed and bushy tailed and brimming with ideas?

Dilbert.com

You see as well as the usual day-to-day sys admin duties and the constant demands of 'other projects', implementing a new system takes time. Time to understand how things work. Time to read what everyone else is doing. Time to piece it all together with the other systems. Time to get it right.

So the truth be told is that we haven't got as far as you might think. On the other hand we do now have some initial and development Hyper-V hosts running, System Center Virtual Machine Manager 2008 R2 (SCVMM) installed and configured, and 2 big Dell servers waiting to be commissioned as the pukka Hyper-V service.

So why did it take so long to get this far or is this far enough?

When as a team we started looking at virtualization we weren't really sure what we wanted it for, but as the issue hotted up, the technology get better and the vendor's got more plentiful. So we started pushing the boundaries.

As well as the Hyper-V solution the other half of the team have been quietly working away with a product called Virtuozzo, which has been very successful. The way in which is works is very good for deploying hosted solutions. So we pushed this in a way to make the deployment of Terminal Services much easier and quickly expandable. Of course it's not free and it did require us to take on a new team member to help implement, but it has made our Terminal Servers farm much easier to manage. They're also on phase 2 now, rolling up to Windows 2008 Terminal Servers.

So why have two virtualization projects on the go?  Why not? It's all about using the best product for the job. By using both Vituozzo and Hyper-V we get the best of both worlds and manage our total cost of ownership.

Virtualization is here to stay and once we get our heads around clustering Hyper-V with the NetApp filer and rolling out a Windows 2008 Terminal Service, we'll have a very strong solution that can fit any need, easily expand with time, save us money and hopefully allow us to start working on the next project. Maybe?

The blog has landed

  

📥  Windows Team

Cold High

In an effort to keep up with the times and open up communication, this is the brand new blog of the University of Bath Computing Services Windows Team! It's a bit of a mouthful, so just call us the Windows Team for short.

For sure it'll a take a while for some blogs to appear, but here's hoping you find what we have to say informative and educational.

Be sure to pop in from time to time to check in on us and post a comment or two.