Category Archives: Hyper-V

95 Best Practices for Optimizing Hyper-V Performance

We can never get enough performance. Everything needs to be faster, faster, faster! You can find any number of articles about improving Hyper-V performance and best practices, of course, unfortunately, a lot of the information contains errors, FUD, and misconceptions. Some are just plain dated. Technology has changed and experience is continually teaching us new insights. From that, we can build a list of best practices that will help you to tune your system to provide maximum performance.

How to boost hyper-V performance - 95 best practices

Philosophies Used in this Article

This article focuses primarily on performance. It may deviate from other advice that I’ve given in other contexts. A system designed with performance in mind will be built differently from a system with different goals. For instance, a system that tries to provide high capacity at a low price point would have a slower performance profile than some alternatives.

  • Subject matter scoped to the 2012 R2 and 2016 product versions.
  • I want to stay on target by listing the best practices with fairly minimal exposition. I’ll expand ideas where I feel the need; you can always ask questions in the comments section.
  • I am not trying to duplicate pure physical performance in a virtualized environment. That’s a wasted effort.
  • I have already written an article on best practices for balanced systems. It’s a bit older, but I don’t see anything in it that requires immediate attention. It was written for the administrator who wants reasonable performance but also wants to stay under budget.
  • This content targets datacenter builds. Client Hyper-V will follow the same general concepts with variable applicability.

General Host Architecture

If you’re lucky enough to be starting in the research phase — meaning, you don’t already have an environment — then you have the most opportunity to build things properly. Making good purchase decisions pays more dividends than patching up something that you’ve already got.

  1. Do not go in blind.
    • Microsoft Assessment and Planning Toolkit will help you size your environment: MAP Toolkit
    • Ask your software vendors for their guidelines for virtualization on Hyper-V.
    • Ask people that use the same product(s) if they have virtualized on Hyper-V.
  2. Stick with logo-compliant hardware. Check the official list:
  3. Most people will run out of memory first, disk second, CPU third, and network last. Purchase accordingly.
  4. Prefer newer CPUs, but think hard before going with bleeding edge. You may need to improve performance by scaling out. Live Migration requires physical CPUs to be the same or you’ll need to enable CPU compatibility mode. If your environment starts with recent CPUs, then you’ll have the longest amount of time to be able to extend it. However, CPUs commonly undergo at least one revision, and that might be enough to require compatibility mode. Attaining maximum performance may reduce virtual machine mobility.
  5. Set a target density level, e.g. “25 virtual machines per host”. While it may be obvious that higher densities result in lower performance, finding the cut-off line for “acceptable” will be difficult. However, having a target VM number in mind before you start can make the challenge less nebulous.
  6. Read the rest of this article before you do anything.

Management Operating System

Before we carry on, I just wanted to make sure to mention that Hyper-V is a type 1 hypervisor, meaning that it runs right on the hardware. You can’t “touch” Hyper-V because it has no direct interface. Instead, you install a management operating system and use that to work with Hyper-V. You have three choices:

Note: Nano Server initially offered Hyper-V, but that functionality will be removed (or has already been removed, depending on when you read this). Most people ignore the fine print of using Nano Server, so I never recommended it anyway.

TL;DR: In absence of a blocking condition, choose Hyper-V Server. A solid blocking condition would be the Automatic Virtual Machine Activation feature of Datacenter Edition. In such cases, the next preferable choice is Windows Server in Core mode.

I organized those in order by distribution size. Volumes have been written about the “attack surface” and patching. Most of that material makes me roll my eyes. No matter what you think of all that, none of it has any meaningful impact on performance. For performance, concern yourself with the differences in CPU and memory footprint. The widest CPU/memory gap lies between Windows Server and Windows Server Core. When logged off, the Windows Server GUI does not consume many resources, but it does consume some. The space between Windows Server Core and Hyper-V Server is much tighter, especially when the same features/roles are enabled.

One difference between Core and Hyper-V Server is the licensing mechanism. On Datacenter Edition, that does include the benefit of Automatic Virtual Machine Activation (AVMA). That only applies to the technological wiring. Do not confuse it with the oft-repeated myth that installing Windows Server grants guest licensing privileges. The legal portion of licensing stands apart; read our eBook for starting information.

Because you do not need to pay for the license for Hyper-V Server, it grants one capability that Windows Server does not: you can upgrade at any time. That allows you to completely decouple the life cycle of your hosts from your guests. Such detachment is a hallmark of the modern cloud era.

If you will be running only open source operating systems, Hyper-V Server is the natural choice. You don’t need to pay any licensing fees to Microsoft at all with that usage. I don’t realistically expect any pure Linux shops to introduce a Microsoft environment, but Linux-on-Hyper-V is a fantastic solution in a mixed-platform environment. And with that, let’s get back onto the list.

Management Operating System Best Practices for Performance

  1. Prefer Hyper-V Server first, Windows Server Core second
  2. Do not install any software, feature, or role in the management operating system that does not directly aid the virtual machines or the management operating system. Hyper-V prioritizes applications in the management operating system over virtual machines. That’s because it trusts you; if you are running something in the management OS, it assumes that you really need it.
  3. Do not log on to the management operating system. Install the management tools on your workstation and manipulate Hyper-V remotely.
  4. If you must log on to the management operating system, log off as soon as you’re done.
  5. Do not browse the Internet from the management operating system. Don’t browse from any server, really.
  6. Stay current on mainstream patches.
  7. Stay reasonably current on driver versions. I know that many of my peers like to install drivers almost immediately upon release, but I can’t join that camp. While it’s not entirely unheard of for a driver update to bring performance improvements, it’s not common. With all of the acquisitions and corporate consolidations going on in the hardware space — especially networking — I feel that the competitive drive to produce quality hardware and drivers has entered a period of decline. In simple terms, view new drivers as a potential risk to stability, performance, and security.
  8. Join your hosts to the domain. Systems consume less of your time if they answer to a central authority.
  9. Use antivirus and intrusion prevention. As long you choose your anti-malware vendor well and the proper exclusions are in place, performance will not be negatively impacted. Compare that to the performance of a compromised system.
  10. Read through our article on host performance tuning.

Leverage Containers

In the “traditional” virtualization model, we stand up multiple virtual machines running individual operating system environments. As “virtual machine sprawl” sets in, we wind up with a great deal of duplication. In the past, we could justify that as a separation of the environment. Furthermore, some Windows Server patches caused problems for some software but not others. In the modern era, containers and omnibus patch packages have upset that equation.

Instead of building virtual machine after virtual machine, you can build a few virtual machines. Deploy containers within them. Strategies for this approach exceed the parameters of this article, but you’re aiming to reduce the number of disparate complete operating system environments deployed. With careful planning, you can reduce density while maintaining a high degree of separation for your services. Fewer kernels are loaded, fewer context switches occur, less memory contains the same code bits, fewer disk seeks to retrieve essentially the same information from different locations.

  1. Prefer containers over virtual machines where possible.


You can’t do a great deal to tune CPU performance in Hyper-V. Overall, I count that among my list of “good things”; Microsoft did the hard work for you.

  1. Follow our article on host tuning; pay special attention to C States and the performance power settings.
  2. For Intel chips, leave hyperthreading on unless you have a defined reason to turn it off.
  3. Leave NUMA enabled in hardware. On your VMs’ property sheet, you’ll find a Use Hardware Topology button. Remember to use that any time that you adjust the number of vCPUs assigned to a virtual machine or move it to a host that has a different memory layout (physical core count and/or different memory distribution).
    best pratices for optimizing hyper-v performance - settings NUMA configuration
  4. Decide whether or not to allow guests to span NUMA nodes (the global host NUMA Spanning setting). If you size your VMs to stay within a NUMA node and you are careful to not assign more guests than can fit solidly within each NUMA node, then you can increase individual VM performance. However, if the host has trouble locking VMs into nodes, then you can negatively impact overall memory performance. If you’re not sure, just leave NUMA at defaults and tinker later.
  5. For modern guests, I recommend that you use at least two virtual CPUs per virtual machine. Use more in accordance with the virtual machine’s performance profile or vendor specifications. This is my own personal recommendation; I can visibly detect the response difference between a single vCPU guest and a dual vCPU guest.
  6. For legacy Windows guests (Windows XP/Windows Server 2003 and earlier), use 1 vCPU. More will likely hurt performance more than help.
  7. Do not grant more than 2 vCPU to a virtual machine without just cause. Hyper-V will do a better job reducing context switches and managing memory access if it doesn’t need to try to do too much core juggling. I’d make exceptions for very low-density hosts where 2 vCPU per guest might leave unused cores. At the other side, if you’re assigning 24 cores to every VM just because you can, then you will hurt performance.
  8. If you are preventing VMs from spanning NUMA nodes, do not assign more vCPU to a VM than you have matching physical cores in a NUMA node (usually means the number of cores per physical socket, but check with your hardware manufacturer).
  9. Use Hyper-V’s priority, weight, and reservation settings with great care. CPU bottlenecks are highly uncommon; look elsewhere first. A poor reservation will cause more problems than it solves.


I’ve long believed that every person that wants to be a systems administrator should be forced to become conversant in x86 assembly language, or at least C. I can usually spot people that have no familiarity with programming in such low-level languages because they almost invariably carry a bizarre mental picture of how computer memory works. Fortunately, modern memory is very, very, very fast. Even better, the programmers of modern operating system memory managers have gotten very good at their craft. Trying to tune memory as a systems administrator rarely pays dividends. However, we can establish some best practices for memory in Hyper-V.

  1. Follow our article on host tuning. Most importantly, if you have multiple CPUs, install your memory such that it uses multi-channel and provides an even amount of memory to each NUMA node.
  2. Be mindful of operating system driver quality. Windows drivers differ from applications in that they can permanently remove memory from the available pool. If they do not properly manage that memory, then you’re headed for some serious problems.
  3. Do not make your CSV cache too large.
  4. For virtual machines that will perform high quantities of memory operations, avoid dynamic memory. Dynamic memory disables NUMA (out of necessity). How do you know what constitutes a “high volume”? Without performance monitoring, you don’t.
  5. Set your fixed memory VMs to a higher priority and a shorter startup delay than your Dynamic Memory VMs. This ensures that they will start first, allowing Hyper-V to plot an optimal NUMA layout and reduce memory fragmentation. It doesn’t help a lot in a cluster, unfortunately. However, even in the best case, this technique won’t yield many benefits.
  6. Do not use more memory for a virtual machine than you can prove that it needs. Especially try to avoid using more memory than will fit in a single NUMA node.
  7. Use Dynamic Memory for virtual machines that do not require the absolute fastest memory performance.
  8. For Dynamic Memory virtual machines, pay the most attention to the startup value. It sets the tone for how the virtual machine will be treated during runtime. For virtual machines running full GUI Windows Server, I tend to use a startup of either 1 GB or 2 GB, depending on the version and what else is installed.
  9. For Dynamic Memory VMs, set the minimum to the operating system vendor’s stated minimum (512 MB for Windows Server). If the VM hosts a critical application, add to the minimum to ensure that it doesn’t get choked out.
  10. For Dynamic Memory VMs, set the maximum to a reasonable amount. You’ll generally discover that amount through trial and error and performance monitoring. Do not set it to an arbitrarily high number. Remember that, even on 2012 R2, you can raise the maximum at any time.

Check the CPU section for NUMA guidance.


In the time that I’ve been helping people with Hyper-V, I don’t believe that I’ve seen anyone waste more time worrying about anything that’s less of an issue than networking. People will read whitepapers and forums and blog articles and novels and work all weekend to draw up intricately designed networking layouts that need eight pages of documentation. But, they won’t spend fifteen minutes setting up a network utilization monitor. I occasionally catch grief for using MRTG since it’s old and there are shinier, bigger, bolder tools, but MRTG is easy and quick to set up. You should know how much traffic your network pushes. That knowledge can guide you better than any abstract knowledge or feature list.

That said, we do have many best practices for networking performance in Hyper-V.

  1. Follow our article on host tuning. Especially pay attention to VMQ on gigabit and separation of storage traffic.
  2. If you need your network to go faster, use faster adapters and switches. A big team of gigabit won’t keep up with a single 10 gigabit port.
  3. Use a single virtual switch per host. Multiple virtual switches add processing overhead. Usually, you can get a single switch to do whatever you wanted multiple switches to do.
  4. Prefer a single large team over multiple small teams. This practice can also help you to avoid needless virtual switches.
  5. For gigabit, anything over 4 physical ports probably won’t yield meaningful returns. I would use 6 at the outside. If you’re using iSCSI or SMB, then two more physical adapters just for that would be acceptable.
  6. For 10GbE, anything over 2 physical ports probably won’t yield meaningful returns.
  7. If you have 2 10GbE and a bunch of gigabit ports in the same host, just ignore the gigabit. Maybe use it for iSCSI or SMB, if it’s adequate for your storage platform.
  8. Make certain that you understand how the Hyper-V virtual switch functions. Most important:
    • You cannot “see” the virtual switch in the management OS except with Hyper-V specific tools. It has no IP address and no presence in the Network and Sharing Center applet.
    • Anything that appears in Network and Sharing Center that you think belongs to the virtual switch is actually a virtual network adapter.
    • Layer 3 (IP) information in the host has no bearing on guests — unless you create an IP collision
  9. Do not create a virtual network adapter in the management operating system for the virtual machines. I did that before I understood the Hyper-V virtual switch, and I have encountered lots of other people that have done it. The virtual machines will use the virtual switch directly.
  10. Do not multi-home the host unless you know exactly what you are doing. Valid reasons to multi-home:
    • iSCSI/SMB adapters
    • Separate adapters for cluster roles. e.g. “Management”, “Live Migration”, and “Cluster Communications”
  11. If you multi-home the host, give only one adapter a default gateway. If other adapters must use gateways, use the old route command or the new New-NetRoute command.
  12. Do not try to use internal or private virtual switches for performance. The external virtual switch is equally fast. Internal and private switches are for isolation only.
  13. If all of your hardware supports it, enable jumbo frames. Ensure that you perform validation testing (i.e.:
    ping storage-ip -f -l 8000)
  14. Pay attention to IP addressing. If traffic needs to locate an external router to reach another virtual adapter on the same host, then traffic will traverse the physical network.
  15. Use networking QoS if you have identified a problem.
    • Use datacenter bridging, if your hardware supports it.
    • Prefer the Weight QoS mode for the Hyper-V switch, especially when teaming.
    • To minimize the negative side effects of QoS, rely on limiting the maximums of misbehaving or non-critical VMs over trying to guarantee minimums for vital VMs.
  16. If you have SR-IOV-capable physical NICs, it provides the best performance. However, you can’t use the traditional Windows team for the physical NICs. Also, you can’t use VMQ and SR-IOV at the same time.
  17. Switch-embedded teaming (2016) allows you to use SR-IOV. Standard teaming does not.
  18. If using VMQ, configure the processor sets correctly.
  19. When teaming, prefer Switch Independent mode with the Dynamic load balancing algorithm. I have done some performance testing on the types (near the end of the linked article). However, a reader commented on another article that the Dynamic/Switch Independent combination can cause some problems for third-party load balancers (see comments section).


When you need to make real differences in Hyper-V’s performance, focus on storage. Storage is slow. The best way to make storage not be slow is to spend money. But, we have other ways.

  1. Follow our article on host tuning. Especially pay attention to:
    • Do not break up internal drive bays between Hyper-V and the guests. Use one big array.
    • Do not tune the Hyper-V partition for speed. After it boots, Hyper-V averages zero IOPS for itself. As a prime example, don’t put Hyper-V on SSD and the VMs on spinning disks. Do the opposite.
    • The best ways to get more storage speed is to use faster disks and bigger arrays. Almost everything else will only yield tiny differences.
  2. For VHD (not VHDX), use fixed disks for maximum performance. Dynamically-expanding VHD is marginally, but measurably, slower.
  3. For VHDX, use dynamically-expanding disks for everything except high-utilization databases. I receive many arguments on this, but I’ve done the performance tests and have years of real-world experience. You can trust that (and run the tests yourself), or you can trust theoretical whitepapers from people that make their living by overselling disk space but have perpetually misplaced their copy of diskspd.
  4. Avoid using shared VHDX (2012 R2) or VHDS (2016). Performance still isn’t there. Give this technology another maturation cycle or two and look at it again.
  5. Where possible, do not use multiple data partitions in a single VHD/X.
  6. When using Cluster Shared Volumes, try to use at least as many CSVs as you have nodes. Starting with 2012 R2, CSV ownership will be distributed evenly, theoretically improving overall access.
  7. You can theoretically improve storage performance by dividing virtual machines across separate storage locations. If you need to make your arrays span fewer disks in order to divide your VMs’ storage, you will have a net loss in performance. If you are creating multiple LUNs or partitions across the same disks to divide up VMs, you will have a net loss in performance.
  8. For RDS virtual machine-based VDI, use hardware-based or Windows’ Hyper-V-mode deduplication on the storage system. The read hits, especially with caching, yield positive performance benefits.
  9. The jury is still out on using host-level deduplication for Windows Server guests, but it is supported with 2016. I personally will be trying to place Server OS disks on SMB storage deduplicated in Hyper-V mode.
  10. The slowest component in a storage system is the disk(s); don’t spend a lot of time worrying about controllers beyond enabling caching.
  11. RAID-0 is the fastest RAID type, but provides no redundancy.
  12. RAID-10 is generally the fastest RAID type that provides redundancy.
  13. For Storage Spaces, three-way mirror is fastest (by a lot).
  14. For remote storage, prefer MPIO or SMB multichannel over multiple unteamed adapters. Avoid placing this traffic on teamed adapters.
  15. I’ve read some scattered notes that say that you should format with 64 kilobyte allocation units. I have never done this, mostly because I don’t think about it until it’s too late. If the default size hurts anything, I can’t tell. Someday, I’ll remember to try it and will update this article after I’ve gotten some performance traces. If you’ll be hosting a lot of SQL VMs and will be formatting their VHDX with 64kb AUs, then you might get more benefit.
  16. I still don’t think that ReFS is quite mature enough to replace NTFS for Hyper-V. For performance, I definitely stick with NTFS.
  17. Don’t do full defragmentation. It doesn’t help. The minimal defragmentation that Windows automatically performs is all that you need. If you have some crummy application that makes this statement false, then stop using that application or exile it to its own physical server. Defragmentation’s primary purpose is to wear down your hard drives so that you have to buy more hard drives sooner than necessary, which is why employees of hardware vendors recommend it all the time. If you have a personal neurosis that causes you pain when a disk becomes “too” fragmented, use Storage Live Migration to clear and then re-populate partitions/LUNs. It’s wasted time that you’ll never get back, but at least it’s faster. Note: All retorts must include verifiable and reproducible performance traces, or I’m just going to delete them.


For real performance, don’t cluster virtual machines. Use fast internal or direct-attached SSDs. Cluster for redundancy, not performance. Use application-level redundancy techniques instead of relying on Hyper-V clustering.

In the modern cloud era, though, most software doesn’t have its own redundancy and host clustering is nearly a requirement. Follow these best practices:

  1. Validate your cluster. You may not need to fix every single warning, but be aware of them.
  2. Follow our article on host tuning. Especially pay attention to the bits on caching storage. It includes a link to enable CSV caching.
  3. Remember your initial density target. Add as many nodes as necessary to maintain that along with sufficient extra nodes for failure protection.
  4. Use the same hardware in each node. You can mix hardware, but CPU compatibility mode and mismatched NUMA nodes will have at least some impact on performance.
  5. For Hyper-V, every cluster node should use a minimum of two separate IP endpoints. Each IP must exist in a separate subnet. This practice allows the cluster to establish multiple simultaneous network streams for internode traffic.
    • One of the addresses must be designated as a “management” IP, meaning that it must have a valid default gateway and register in DNS. Inbound connections (such as your own RDP and PowerShell Remoting) will use that IP.
    • None of the non-management IPs should have a default gateway or register in DNS.
    • One alternative IP endpoint should be preferred for Live Migration. Cascade Live Migration preference order through the others, ending with the management IP. You can configure this setting most easily in Failover Cluster Manager by right-clicking on the Networks node.
    • Further IP endpoints can be used to provide additional pathways for cluster communications. Cluster communications include the heartbeat, cluster status and update messages, and Cluster Shared Volume information and Redirected Access traffic.
    • You can set any adapter to be excluded from cluster communications but included in Live Migration in order to enforce segregation. Doing so generally does not improve performance, but may be desirable in some cases.
    • You can use physical or virtual network adapters to host cluster IPs.
    • The IP for each cluster adapter must exist in a unique subnet on that host.
    • Each cluster node must contain an IP address in the same subnet as the IPs on other nodes. If a node does not contain an IP in a subnet that exists on other nodes, then that network will be considered “partitioned” and the node(s) without a member IP will be excluded from that network.
    • If the host will connect to storage via iSCSI, segregate iSCSI traffic onto its own IP(s). Exclude it/them from cluster communications and Live Migration. Because they don’t participate in cluster communications, it is not absolutely necessary that they be placed into separate subnets. However, doing so will provide some protection from network storms.
  6. If you do not have RDMA-capable physical adapters, Compression usually provides the best Live Migration performance.
  7. If you do have RDMA-capable physical adapters, SMB usually provides the best Live Migration performance.
  8. I don’t recommend spending time tinkering with the metric to shape CSV traffic anymore. It utilizes SMB, so the built-in SMB multi-channel technology can sort things out.

Virtual Machines

The preceding guidance obliquely covers several virtual machine configuration points (check the CPU and the memory sections). We have a few more:

  1. Don’t use Shielded VMs or BitLocker. The encryption and VMWP hardening incur overhead that will hurt performance. The hit is minimal — but this article is about performance.
  2. If you have 1) VMs with very high inbound networking needs, 2) physical NICs >= 10GbE, 3) VMQ enabled, 4) spare CPU cycles, then enable RSS within the guest operating systems. Do not enable RSS in the guest OS unless all of the preceding are true.
  3. Do not use the legacy network adapter in Generation 1 VMs any more than absolutely necessary.
  4. Utilize checkpoints rarely and briefly. Know the difference between standard and “production” checkpoints.
  5. Use time synchronization appropriately. Meaning, virtual domain controllers should not have the Hyper-V time synchronization service enabled, but all other VMs should (generally speaking). The hosts should pull their time from the domain hierarchy. If possible, the primary domain controller should be pulling from a secured time source.
  6. Keep Hyper-V guest services up-to-date. Supported Linux systems can be updated via kernel upgrades/updates from their distribution repositories. Windows 8.1+ and Windows Server 2012 R2+ will update from Windows Update.
  7. Don’t do full defragmentation in the guests, either. Seriously. We’re administering multi-spindle server equipment here, not displaying a progress bar to someone with a 5400-RPM laptop drive so that they feel like they’re accomplishing something.
  8. If the virtual machine’s primary purpose is to run an application that has its own replication technology, don’t use Hyper-V Replica. Examples: Active Directory and Microsoft SQL Server. Such applications will replicate themselves far more efficiently than Hyper-V Replica.
  9. If you’re using Hyper-V Replica, consider moving the VMs’ page files to their own virtual disk and excluding it from the replica job. If you have a small page file that doesn’t churn much, that might cost you more time and effort than you’ll recoup.
  10. If you’re using Hyper-V Replica, enable compression if you have spare CPU but leave it disabled if you have spare network. If you’re not sure, use compression.
  11. If you are shipping your Hyper-V Replica traffic across an encrypted VPN or keeping its traffic within secure networks, use Kerberos. SSL en/decryption requires CPU. Also, the asymmetrical nature of SSL encryption causes the encrypted data to be much larger than its decrypted source.


You must monitor your systems. Monitoring is not and has never been, an optional activity.

  1. Be aware of Hyper-V-specific counters. Many people try to use Task Manager in the management operating system to gauge guest CPU usage, but it just doesn’t work. The management operating system is a special-case virtual machine, which means that it is using virtual CPUs. Its Task Manager cannot see what the guests are doing.
  2. Performance Monitor has the most power of any built-in tool, but it’s tough to use. Look at something like Performance Analysis of Logs (PAL) tool, which understands Hyper-V.
  3. In addition to performance monitoring, employ state monitoring. With that, you no longer have to worry (as much) about surprise events like disk space or memory filling up. I like Nagios, as regular readers already know, but you can select from many packages.
  4. Take periodic performance baselines and compare them to earlier baselines

If you’re able to address a fair proportion of points from this list, I’m sure you’ll see a boost in Hyper-V performance. Don’t forget this list is not exhaustive and I’ll be adding to it periodically to ensure it’s as comprehensive as possible however if you think there’s something missing, let me know in the comments below and you may see the number 95 increase!

Container Images are now out for Windows Server version 1709!

With the release of Windows Server version 1709 also come Windows Server Core and Nano Server base OS container images.

It is important to note that while older versions of the base OS container images will work on a newer host (with Hyper-V isolation), the opposite is not true. Container images based on Windows Server version 1709 will not work on a host using Windows Server 2016.  Read more about the different versions of Windows Server.

We’ve also made some changes to our tagging scheme so you can more easily specify which version of the container images you want to use.  From now on, the “latest” tag will follow the releases of the current LTSC product, Windows Server 2016. If you want to keep up with the latest patches for Windows Server 2016, you can use:


in your dockerfiles to get the most up-to-date version of the Windows Server 2016 base OS images. You can also continue using specific versions of the Windows Server 2016 base OS container images by using the tags specifying the build, like so:


If you would like to use base OS container images based on Windows Server version 1709, you will have to specify that with the tag. In order to get the most up-to-date base OS container images of Windows Server version 1709, you can use the tags:


And if you would like a specific version of these base OS container images, you can specify the KB number that you need on the tag, like this:


We hope that this tagging scheme will ensure that you always choose the image that you want and need for your environment. Please let us know in the comments if you have any feedback for us.

Note: We currently do not intend to use the build numbers to specify Windows Server version 1709 container images. We will only be using the KB schema specified above for the tagging of these images. Let us know if you have feedback about this as well


How to Perform Hyper-V Storage Migration

New servers? New SAN? Trying out hyper-convergence? Upgrading to Hyper-V 2016? Any number of conditions might prompt you to move your Hyper-V virtual machine’s storage to another location. Let’s look at the technologies that enable such moves.

An Overview of Hyper-V Migration Options

Hyper-V offers numerous migration options. Each has its own distinctive features. Unfortunately, we in the community often muck things up by using incorrect and confusing terminology. So, let’s briefly walk through the migration types that Hyper-V offers:

  • Quick migration: Cluster-based virtual machine migration that involves placing a virtual machine into a saved state, transferring ownership to another node in the same cluster, and resuming the virtual machine. A quick migration does not involve moving anything that most of us consider storage.
  • Live migration: Cluster-based virtual machine migration that involves transferring the active state of a running virtual machine to another node in the same cluster. A Live Migration does not involve moving anything that most of us consider storage.
  • Storage migration: Any technique that utilizes the Hyper-V management service to relocate any file-based component that belongs to a virtual machine. This article focuses on this migration type, so I won’t expand any of those thoughts in this list.
  • Shared Nothing Live Migration: Hyper-V migration technique between two hosts that does not involve clustering. It may or may not include a storage migration. The virtual machine might or might not be running. However, this migration type always includes ownership transfer from one host to another.

It Isn’t Called Storage Live Migration

I have always called this operation “Storage Live Migration”. I know lots of other authors call it “Storage Live Migration”. But, Microsoft does not call it “Storage Live Migration”. They just call it “Storage Migration”. The closest thing that I can find to “Storage Live Migration” in anything from Microsoft is a 2012 TechEd recording by Benjamin Armstrong. The title of that presentation includes the phrase “Live Storage Migration”, but I can’t determine if the “Live” just modifies “Storage Migration” or if Ben uses it as part of the technology name. I suppose I could listen to the entire hour and a half presentation, but I’m lazy. I’m sure that it’s a great presentation, if anyone wants to listen and report back.

Anyway, does it matter? I don’t really think so. I’m certainly not going to correct anyone that uses that phrase. However, the virtual machine does not necessarily need to be live. We use the same tools and commands to move a virtual machine’s storage whether it’s online or offline. So, “Storage Migration” will always be a correct term. “Storage Live Migration”, not so much. However, we use the term “Shared Nothing Live Migration” for virtual machines that are turned off, so we can’t claim any consistency.

What Can Be Moved with Hyper-V Storage Migration?

When we talk about virtual machine storage, most people think of the places where the guest operating system stores its data. That certainly comprises the physical bulk of virtual machine storage. However, it’s also only one bullet point on a list of multiple components that form a virtual machine.

Independently, you can move any of these virtual machine items:

  • The virtual machine’s core files (configuration in xml or .vmcx, .bin, .vsv, etc.)
  • The virtual machine’s checkpoints (essentially the same items as the preceding bullet point, but for the checkpoint(s) instead of the active virtual machine)
  • The virtual machine’s second-level paging file location. I have not tested to see if it will move a VM with active second-level paging files, but I have no reason to believe that it wouldn’t
  • Virtual hard disks attached to a virtual machine
  • ISO images attached to a virtual machine

We most commonly move all of these things together. Hyper-V doesn’t require that, though. Also, we can move all of these things in the same operation but distribute them to different destinations.

What Can’t Be Moved with Hyper-V Storage Migration?

In terms of storage, we can move everything related to a virtual machine. But, we can’t move the VM’s active, running state with Storage Migration. Storage Migration is commonly partnered with a Live Migration in the operation that we call “Shared Nothing Live Migration”. To avoid getting bogged down in implementation details that are more academic than practical, just understand one thing: when you pick the option to move the virtual machine’s storage, you are not changing which Hyper-V host owns and runs the virtual machine.

More importantly, you can’t use any Microsoft tool-based technique to separate a differencing disk from its parent. So, if you have an AVHDX (differencing disk created by the checkpointing mechanism) and you want to move it away from its source VHDX, Storage Migration will not do it. If you instruct Storage Migration to move the AVHDX, the entire disk chain goes along for the ride.

Uses for Hyper-V Storage Migration

Out of all the migration types, storage migration has the most applications and special conditions. For instance, Storage Migration is the only Hyper-V migration type that does not always require domain membership. Granted, the one exception to the domain membership rule won’t be very satisfying for people that insist on leaving their Hyper-V hosts in insecure workgroup mode, but I’m not here to please those people. I’m here to talk about the nuances of Storage Migration.

Local Relocation

Let’s start with the simplest usage: relocation of local VM storage. Some situations in this category:

  • You left VMs in the default “C:ProgramDataMicrosoftWindowsHyper-V” and/or “C:UsersPublicDocumentsHyper-VVirtual Hard Disks” locations and you don’t like it
  • You added new internal storage as a separate volume and want to re-distribute your VMs
  • You have storage speed tiers but no active management layer
  • You don’t like the way your VMs’ files are laid out
  • You want to defragment VM storage space. It’s a waste of time, but it works.

Network Relocation

With so many ways to do network storage, it’s nearly a given that we’ll all need to move a VHDX across ours at some point. Some situations:

  • You’re migrating from local storage to network storage
  • You’re replacing a SAN or NAS and need to relocate your VMs
  • You’ve expanded your network storage and want to redistribute your VMs

Most of the reasons listed under “Local Relocation” can also apply to network relocation.

Cluster Relocation

We can’t always build our clusters perfectly from the beginning. For the most part, a cluster’s relocation needs list will look like the local and network lists above. A few others:

  • Your cluster has new Cluster Shared Volumes that you want to expand into
  • Existing Cluster Shared Volumes do not have a data distribution that does not balance well. Remember that data access from a CSV owner node is slightly faster than from a non-owner node

The reasons matter less than the tools when you’re talking about clusters. You can’t use the same tools and techniques to move virtual machines that are protected by Failover Clustering under Hyper-V as you use for non-clustered VMs.

Turning the VM Off Makes a Difference for Storage Migration

You can perform a very simple experiment: perform a Storage Migration for a virtual machine while it’s on, then turn it off and migrate it back. The virtual machine will move much more quickly while it’s off. This behavior can be explained in one word: synchronization.

When the virtual machine is off, a Storage Migration is essentially a monitored file copy. The ability of the constituent parts to move bits from source to destination sets the pace of the move. When the virtual machine is on, all of the rules change. The migration is subjected to these constraints:

  • The virtual machine’s operating system must remain responsive
  • Writes must be properly captured
  • Reads must occur from the most appropriate source

Even if the guest operating does not experience much activity during the move, that condition cannot be taken as a constant. In other words, Hyper-V needs to be ready for it to start demanding lots of I/O at any time.

So, the Storage Migration of a running virtual machine will always take longer than the Storage Migration of a virtual machine in an off or saved state. You can choose the convenience of an online migration or the speed of an offline migration.

Note: You can usually change a virtual machine’s power state during a Storage Migration. It’s less likely to work if you are moving across hosts.

How to Perform Hyper-V Storage Migration with PowerShell

The nice thing about using PowerShell for Storage Migration: it works for all Storage Migration types. The bad thing about using PowerShell for Storage Migration: it can be difficult to get all of the pieces right.

The primary cmdlet to use is Move-VMStorage. If you will be performing a Shared Nothing Live Migration, you can also use Move-VM. The parts of Move-VM that pertain to storage match Move-VMStorage. Move-VM has uses, requirements, and limitations that don’t pertain to the topic of this article, so I won’t cover Move-VM here.

A Basic Storage Migration in PowerShell

Let’s start with an easy one. Use this when you just want all of a VM’s files to be in one place:

This will move the virtual machine named testvm so that all of its components reside under the C:LocalVMs folder. That means:

  • The configuration files will be placed in C:LocalVMsVirtual Machines
  • The checkpoint files will be placed in C:LocalVMsSnapshots
  • The VHDXs will be placed in C:LocalVMsVirtual Hard Disks
  • Depending on your version, an UndoLog Configuration folder will be created if it doesn’t already exist. The folder is meant to contain Hyper-V Replica files. It may be created even for virtual machines that aren’t being replicated.

Complex Storage Migrations in PowerShell

For more complicated move scenarios, you won’t use the DestinationStoragePath parameter. You’ll use one or more of the individual component parameters. Choose from the following:

  • VirtualMachinePath: Where to place the VM’s configuration files.
  • SnapshotFilePath: Where to place the VM’s checkpoint files (again, NOT the AVHDXs!)
  • SmartPagingFilePath: Where to place the VM’s smart paging files
  • Vhds: An array of hash tables that indicate where to place individual VHD/X files.

Some notes on these items:

  • You are not required to use all of these parameters. If you do not specify a parameter, then its related component is left alone. Meaning, it doesn’t get moved at all.
  • If you’re trying to use this to get away from those auto-created Virtual Machines and Snapshots folders, it doesn’t work. They’ll always be created as sub-folders of whatever you type in.
  • It doesn’t auto-create a Virtual Hard Disks folder.
  • If you were curious whether or not you needed to specify those auto-created subfolders, the answer is: no. Move-VMStorage will always create them for you (unless they already exist).
  • The VHDs hash table is the hardest part of this whole thing. I’m usually a PowerShell-first kind of guy, but even I tend to go to the GUI for Storage Migrations.

The following will move all components except VHDs, which I’ll tackle in the next section:

Move-VMStorage’s Array of Hash Tables for VHDs

The three …FilePath parameters are easy: just specify the path. The Vhds parameter is tougher. It is one or more hash tables inside an array.

First, the hash tables. A hash table is a custom object that looks like an array, but each entry has a unique name. The hash tables that Vhds expects have a SourceFilePath entry and a DestinationFilePath entry. Each must be fully-qualified for a file. A hash table is contained like this: @{ }. The name of an entry and its value are joined with an =. Entries are separated by a ; So, if you want to move the VHDX named svtest.vhdx from \svstoreVMs to C:LocalVMstestvm, you’d use this hash table:

Reading that, you might ask (quite logically): “Can I change the name of the VHDX file when I move it?” The answer: No, you cannot. So, why then do you need to enter the full name of the destination file? I don’t know!

Next, the arrays. An array is bounded by @( ). Its entries are separated by commas. So, to move two VHDXs, you would do something like this:

I broke that onto multiple lines for legibility. You can enter it all on one line. Note where I used parenthesis and where I used curly braces.

Tip: To move a single VHDX file, you don’t need to do the entire array notation. You can use the first example with Vhds.

A Practical Move-VMStorage Example with Vhds

If you’re looking at all that and wondering why you’d ever use PowerShell for such a thing, I have the perfect answer: scripting. Don’t do this by hand. Use it to move lots of VMs in one fell swoop. If you want to see a plain example of the Vhds parameter in action, the Get-Help examples show one. I’ve got a more practical script in mind.

The following would move all VMs on the host. All of their config, checkpoint, and second-level paging files will be placed on a share named “\vmstoreslowstorage”. All of their VHDXs will be placed on a share named “\vmstorefaststorage”. We will have PowerShell deal with the source paths and file names.

I used splatting for the parameters for two reasons: 1, legibility. 2, to handle VMs without any virtual hard disks.

How to Perform Hyper-V Storage Migration with Hyper-V Manager

Hyper-V Manager can only be used for non-clustered virtual machines. It utilizes a wizard format. To use it to move a virtual machine’s storage:

  1. Right-click on the virtual machine and click Move.
  2. Click Next on the introductory page.
  3. Change the selection to Move the virtual machine’s storage (the same storage options would be available if you moved the VM’s ownership, but that’s not part of this article)
  4. Choose how to perform the move. You can move everything to the same location, you can move everything to different locations, or you can move only the virtual hard disks.
  5. What screens you see next will depend on what you chose. We’ll cover each branch.

If you opt to move everything to one location, the wizard will show you this simple page:


If you choose the option to Move the virtual machine’s data to different locations, you will first see this screen:


For every item that you check, you will be given a separate screen where you indicate the desired location for that item. The wizard uses the same screen for these items as it does for the hard-disks only option. I’ll show its screen shot next.

If you choose Move only the virtual machine’s virtual hard disks, then you will be given a sequence of screens where you instruct it where to move the files. These are the same screens used for the individual components from the previous selection:


After you make your selections, you’ll be shown a summary screen where you can click Finish to perform the move:


How to Perform Hyper-V Storage Migration with Failover Cluster Manager

Failover Cluster Manager uses a slick single-screen interface to move storage for cluster virtual machines. To access it, simply right-click a virtual machine, hover over Move, and click Virtual Machine Storage. You’ll see the following screen:


If you just want to move the whole thing to one of the display Cluster Shared Volumes, just drag and drop it down to that CSV in the Cluster Storage heading at the lower left. You can drag and drop individual items or the entire VM. The Destination Folder Path will be populated accordingly.

As you can see in mine, I have all of the components except the VHD on an SMB share. I want to move the VHD to be with the rest. To get a share to show up, click the Add Share button. You’ll get this dialog:


The share will populate underneath the CSVs in the lower left. Now, I can drag and drop that file to the share. View the differences:


Once you have the dialog the way that you like it, click Start.

3 Emerging Technologies that Will Change the Way You Use Hyper-V

Hello once again everyone!

The I.T. landscape changes incredibly quickly (if you know a faster changing industry, I’d love to know!) I.T. professionals need to know what’s coming round the corner to stay ahead of the game or risk being left behind. Well, we don’t want that to happen to you, so we’ve run down what we feel are the three most important emerging technologies that will drastically change the Hyper-V landscape.

  1. Continued Adoption of Public Cloud Platforms – It’s becoming clear that the public cloud is continuing to gain steam. It’s not just one vendor, but several, and it continues to pull workloads from on-premise to the cloud. Many people were keen to wait out this “cloud thing”, but it has become quite clear that it’s here to stay. Capabilities in online platforms such as Microsoft Azure and Amazon AWS, have increasingly made it easier, more cost-effective, and desirable to put workloads in the public cloud. These cloud platforms can often provide services that most customers don’t have available on-premise, and this paired with several other things that we’ll talk about in the webinar are leading to increased adoption of these platforms over on-premise installations.
  2. Azure Stack and the Complete Abstraction of Hyper-V under-the-hood – With some of the latest news and release information out of Microsoft regarding their new Microsoft Azure Stack (MAS), things have taken an interesting turn for Hyper-V. As on-premise administrators have always been used to having direct access to the hypervisor, they may be surprised to learn that Hyper-V is so far under the hood in MAS that you can’t even access it. That’s right. The Hypervisor has become so simplified and automated, that there is no need to directly access it in MAS, but this is primarily because MAS follows the same usage and management guidelines as Microsoft Azure. This will bother a lot of administrators but it’s becoming the world we live in. As such, we’ll be talking about this extensively during the webinar.
  3. Containers and Microservices and why they are a game-changer – Containers has become one of the new buzz-words in the industry. If you’re not aware, you can think of containers as similar to a VM, but fundamentally different. Whereas in a VM you’re virtualizing the OS, and everything on top of it, with containers you’re only virtualizing the application. Much of the underlying support functions are handled by the container host, as opposed to an OS built into a VM. For a long time it seemed that containers were going to primarily be a developer thing, but as the line between IT Pro and Dev continues to blur, Containers can no longer be ignored by IT Pros, and we’ll be talking about that revelation extensively during our panel discussion.

As you can see there is much to talk about, and many will be wondering how this affects them. You’re probably asking yourself questions like: “What new skills should IT Pros be learning to stay relevant?”, “Are hypervisors becoming irrelevant?”, “Will containers replace virtual machines?”, “Is the Cloud here to stay?”, “Is there still a place for Windows Server in the world?”, “What can I do now to stay relevant and what skills do I need to learn to future-proof my career?” Yep, these developments certainly raise a lot of issues which is why we decided to take this topic further.

Curious to know more? Join our Live Webinar!

As you know we love to put on webinars here at Altaro as we find them a critical tool for getting information about new technologies and features to our viewership. We’ve always stuck to the same basic educational format and it’s worked well over the years. However, we’ve always wanted to try something a bit different. There certainly isn’t anything wrong with an educational format, but with some topics, it’s often best to just have a conversation. This idea is at the core of our next webinar along with some critical changes that are occurring within our industry.

For the first time ever, Altaro will be putting on a panel-style webinar with not 1 or 2, but with 3 Microsoft Cloud and Datacenter MVPs. Andy Syrewicze, Didier Van Hoye, and Thomas Maurer will all be hosting this webinar as they talk about some major changes and take your questions/feedback regarding things that are occurring in the industry today. These are things that will affect the way you use and consume Hyper-V.

Webinar Registration

8 Most Important Announcements from Microsoft Ignite 2017

03 Oct 2017 by Andy Syrewicze

Last week saw us close the door on Microsoft Ignite 2017, and while the conference came and went in a blur, there was no lack of information or amazing reveals from Microsoft. While this conference serves as a great way to stay informed on all the new things that Microsoft is working on, I also find that it is a good way to get an overall sense of the company’s overall direction. With that in mind, I wanted to not only talk about some of my favorite reveals from the week but also discuss my take on Microsoft’s overall direction.

Microsoft Ignite 2017 - most important announcements

My take on the week from an Infrastructure Engineering Perspective

To put things simply….. things are changing, and they’re changing in a big way. I’ve had this gut feeling stirring for some time that the way we work with VMs and virtualization was changing, and the week of Ignite was a major confirmation of that. This is not to mention the continued shift from the on-premise model we’re used to, to the new cloud (Public, Private, and Hybrid) model that things are moving too.

It’s very clear that Microsoft is adopting what I would call the “Azure-Everywhere” approach. Sure, you’ve always been able to consume Azure using what Microsoft has publicly available, but things really changed when Azure Stack is put into the mix. Microsoft Azure Stack (MAS) is officially on the market now, and the idea of having MAS in datacenters around the world is an interesting prospect. What I find so interesting about it, is the fact that management of MAS onsite is identical to managing Azure. You use Azure Resource Manager and the same collection of tools to manage both. Pair that with the fact that Hyper-V is so abstracted and under-the-hood in MAS that you can’t even see it, and you’ve got a recipe for major day-to-day changes for infrastructure administrators.

Yes, we’ve still got Windows Server 2016, and the newly announced Honolulu management utility, but If I look out 5, or even 10 years, I’m not sure I see us working with Windows Server anymore in the way that we do so today. I don’t think VM usage will be as prevalent then as it is today either. After last week, I firmly believe that containers will be the “new virtual machine”. I think VMs will stay around for legacy workloads, and for workloads that require additional layers of isolation, but after seeing containers in action last week, I’m all in on that usage model.

We used to see VMs as this amazing cost-reducing technology, and it was for a long time. However, I saw containers do to VMs, what VMs did to physical servers. I attended a session on moving workloads to a container based model, and MetLife was on stage talking about moving some of their infrastructure to containers. In doing so they achieved:

  • -70% reduction in the number of VMs in the environment
  • -67% reduction in needed CPU cores
  • -66% reduction in overall cost of ownership

Those are amazing numbers that nobody can ignore. Given this level of success with containers, I see the industry moving to that deployment model from VMs over the next several years. As much as it pains me to say it, virtual machines are starting to look very “legacy”, and we all need to adjust our skill sets accordingly.

Big Reveals

As you know, Ignite is that time of year where Microsoft makes some fairly large announcements, and below I’ve compiled a list of some of my favorite. While this is by no means a comprehensive list,  but I feel these represent what our readers would find most interesting. Don’t agree? That’s fine! Just let me know what you think were the most important announcements in the comments. Let’s get started.

8. New Azure Exams and Certifications!

With new technologies, come new things to learn, and as such there are 3 new exams on the market today for Azure Technologies.

  • Azure Stack Operators – Exam 537: Configuring and Operating a Hybrid Cloud with Microsoft Azure Stack
  • For Azure Solutions Architects – Exam 539: Managing Linux Workloads on Azure
  • For Azure DevOps – Exam 538: Implementing Microsoft Azure DevOps Solutions

If you’re interested in pursuing any of these (Which I would Highly Recommend) then you can get more information on them at this link.

7. SQL Server 2017 is now Available

Normally I wouldn’t make much of a fuss about SQL Server as I’m not much of a SQL guy myself, but Microsoft did something amazing with this release. SQL Server 2017 will run on Windows, Linux, and inside of Docker Containers. Yes, you read correctly. SQL Server 2017 will run on Linux and inside of docker containers, which opens up a whole new avenue of providing SQL workloads. Exciting times indeed!

6. Patch Management from the Azure Portal

Ever wanted to have WSUS available from the Azure portal? Now you have it. You can easily view, and deploy patches for your Azure based workloads directly from the Azure portal. This includes Linux VMs as well, which is great news as more and more admins are finding themselves managing Linux workloads these days!

5. PowerShell now Available in Azure CLI.

When Azure CLI was announced and released, many people were taken aback at the lack of PowerShell support. This was done for a number of reasons that I won’t get into in this article, but regardless, it has been added in now. It is now possible with Azure CLI to deploy a VM with a single PowerShell cmdlet and more. So, get those scripts ready!

4. Azure File Sync in Preview

I know many friends and colleagues that have been waiting for something like this. You can essentially view this as next-generation DFS. (Though it doesn’t use the same technology). It essentially allows you to sync your on-premise file servers with an Azure Files account for distributed access to the stored information around the globe.

3. Quality of Life Improvements for Windows Containers

While there were no huge reveals in the container space, Windows Server 1709 was announced and contains a lot of improvements and optimizations for running containers on Windows Server. This includes things like smaller images and support for Linux Containers running on Windows Server. I did an Interview with Taylor Brown from the Containers team, which you can view below for more information.

[embedded content]

2. Nested Virtualization in Azure for Production Workloads

Yes, I know, nested virtualization in Azure has been announced for some time. However, what I found different was Microsoft’s Insistence that it could also be used for production workloads. During Scott Guthrie’s Keynote, Corey Sanders actually demonstrated the use of the M-Series (Monster) VM in Azure being used to host production workloads with nested VMs. While not ideal in every scenario obviously, this is simply another tool that we have at our disposal for added flexibility in our day-to-day operations.

If you’re interested, I actually interviewed Rick Claus from the Azure Compute team about this. That Interview can be seen below

[embedded content]

1. Project Honolulu

This one is for the folks that are still strictly interested in only the on-prem stuff. Microsoft revealed and showed us the new Project Honolulu management utility for on-premise workloads. Honolulu, takes the functionality of all the management tools and MMC snap-ins that we’ve been using for years and packages them up into a nice easy to use web-UI. It’s worth a look if you haven’t seen it yet. We even have a nice article on our blog about it if you’re interested in reading more!


As I mentioned, this was by no means a comprehensive list, but we’ll be talking about items (From this list and some not mentioned) from Ignite on our blogs for some time. So, be sure to keep an eye on our blog if you’re interested in more information.

Additionally, if you attended Microsoft Ignite, and you saw a feature or product you think is amazing that is not listed above, be sure to let us know in the comments section below!

Have any questions or feedback?

Leave a comment below!

The Complete Guide to Hyper-V 2016 Integration Services

28 Sep 2017 by Eric Siron
Hyper-V Articles

Isolation represents one of the fundamental features of a hypervisor. If we didn’t want isolation, we would have little need to virtualize anything. We could install one operating system on a host and add software until we ran out of capacity. Security and compatibility prevent that approach. However, too much isolation causes other problems. Microsoft supplies the Hyper-V Guest Services as one solution.

The Isolation Model

Visually, Hyper-V’s isolation model looks something like this:

Hyper-V's isolation model

The management operating system loads the hardware drivers that Hyper-V utilizes, but otherwise, stays separated. Hyper-V keeps the management operating system and virtual machines “partitioned” from itself and each other. Of course, 100% isolation is neither possible nor practical. So, Hyper-V provides several interfaces for the virtual machines to use. VMBus manages most of these interfaces. As a “bus”, it carries requests and responses between Hyper-V and the guests.

The available interfaces in Hyper-V 2016 (that I know of):

  • Emulated and synthetic hardware, SR-IOV, and Discrete Device Access
  • Automatic virtual machine activation (Datacenter Edition only)
  • PowerShell Direct
  • Service monitoring (Failover Clustering only)
  • Integration services
    • Operating system shutdown
    • Time synchronization
    • Data Exchange
    • Heartbeat
    • Backup (volume shadow copy)
    • Guest services
  • Virtual Trusted Platform Module (vTPM)
  • Hyper-V Sockets

This article focuses on the integration services.

An Overview of the Integration Services

Each integration service provides a specific function to the virtual machine. For most, you can figure out its function just by looking at its name. The functions are projected into each virtual machine, but a matching software construct must exist and be enabled within the guest operating system.

For Windows guests, Microsoft provides a set of standard Windows services.

  • Hyper-V Data Exchange Service: aligns with Data Exchange
  • Hyper-V Guest Service Interface: aligns with Guest services
  • Hyper-V Guest Shutdown Service: aligns with Operating system shutdown
  • Hyper-V Time Synchronization Service: aligns with Time synchronization
  • Hyper-V Volume Shadow Copy Requestor: aligns with Backup (volume shadow copy)

Microsoft has distributed these as core components of its operating systems since Windows Vista/Windows Server 2008. Up through Windows Server and Hyper-V Server 2012 R2, every server maintained a local copy of the integration services and conveniently packaged them as vmguest.iso at each reboot. You can find the files in the host’s “Windowsvmguest” folder and the packaged ISO in “WindowsSystem32”.

Starting with Windows 10/Windows Server 2016, these files no longer exist on the host.

Windows Services that are Not Part of the Hyper-V Integration Services

You may see additional services that include Hyper-V or something that implies Hyper-V. Other than the ones that I listed, none are part of the official Hyper-V 2016 Integration Services stack that we’re talking about in this article.

Those services:

  • HV Host Service: Rather than re-invent the wheel, I’ll just paste the perfectly adequate description text right from the service: “Provides an interface for the Hyper-V hypervisor to provide per-partition performance counters to the host operating system.”
  • Hyper-V Host Compute Service: You will find this service on Hyper-V 2016 hosts, including Windows 10 and nested Hyper-V instances. It provides an API for container systems.
  • Hyper-V Remote Desktop Virtualization Service: I have no idea what this service does. It’s always running on my 2016 guests but not on my 2012 R2 guests. My first thought was that it enables Enhanced Session Mode from within the guest. So I stopped it. I had no trouble using Enhanced Session mode after that. It may be for RemoteFx.
  • Hyper-V Virtual Machine Management: This is the familiar VMMS that has been with us since the beginning. In previous versions, it was just called “Virtual Machine Management”. It runs on all Hyper-V hosts and enables VM control functionality (start, stop, etc.).

Integration Services Version Compatibility

Even though the host-side projections exist within every virtual machine that it creates, not every guest operating system has access to the matching client-side service. As an example, the Hyper-V Guest Service Interface cannot be installed into any guest prior to Windows 8.1/Windows Server 2012 R2. Therefore, that service will not be available on earlier operating systems.

Also, the host’s version rules over all guests. If you install a later version on a guest and then move it to an older host, you might have some difficulties.

Installing/Updating Hyper-V Guest Services on Windows

As previously mentioned, all currently supported versions of Windows and Windows Server ship with the guest services already installed. The process for updating them has changed since its inception, however.

For Windows XP/Windows Server 2003 up through Windows 8.1/Windows Server 2012 R2 while running on any Hyper-V host version from 2008 through 2012 R2, you would install/update by inserting the host’s vmguest.iso in the guest. The Hyper-V Virtual Machine connect window included an action:

Integration Services Setup Disk

Any time that Microsoft released an update for the integration services via Windows Update, only the host would be automatically updated. The guests could only be updated via a manual process. You could use the action shown above, manually insert the vmguest.iso, or even manually transfer the install files into the guest.

Now, the client packages for the integration services are also delivered via Windows Update. Simply keep your guests up-to-date and you will no longer need to worry about this.

Note 1: I am not certain what operating systems are included. I know that W8.1/WS2012 R2 and W10/WS2016 receive updates this way. I am not certain about earlier versions. If you know, please tell me.

Note 2: For older operating systems, remember that the host version rules. Meaning, do not attempt to install the files from a 2012 R2 host into a guest running on Hyper-V Server 2008.

Note 3: You’ll face some challenges for unsupported legacy operating systems such as Windows XP. You’ll need to find the newest versions that will still install by trying each of the server versions in turn. Do not expect perfect results.

Installing/Updating Hyper-V Guest Services on Linux

The core Hyper-V guest services have been natively built into the Linux kernel for quite some time. If you use a recent, mainstream distribution, then you’ve already got most of the services. They will automatically be updated anytime you update or upgrade your kernel.

If you wish to update the components separately or on a system with an older kernel, Microsoft makes the Linux Integration Services available for download. As of this writing, the current release version is 4.2. The lis-next GitHub project provides the source for feature backports to older versions. You can watch Microsoft’s official virtualization blog for notifications about new versions.

Not all features are included by default with the kernel-embedded integration services, notably Data Exchange and file copy (the only part of Guest services that Linux supports). Start on Microsoft’s official Linux on Hyper-V page. On the left or at the link list at the bottom, find your distribution’s page. Follow the directions to enable the additional features on your distribution.

Details on the Hyper-V Integration Services

Each service provides specific functionality, described below.

The Hyper-V Operating System Shutdown Service

Hyper-V’s Operating System Shutdown Service communicates with the matching service in the guest operating system to gracefully shut down the guest operating system. Without it, you can only force stop the virtual machine from Hyper-V.

The Hyper-V Time Synchronization Service

Hyper-V’s Time Synchronization Service keeps the guest’s time inline with the host’s. It respects any time zone differences. Do not use it for virtualized domain controllers, especially one that holds the PDC emulator role. Otherwise, it’s generally a good idea to leave it enabled for all guests.

The Hyper-V Data Exchange Service

Hyper-V uses key-value pairs to transmit unformatted information between the guest and the host. On Windows guests, the service operates through the guest’s registry. On Linux, the service operates through specially formatted files. On the host side, the Hyper-V WMI API facilitates key exchange for all guests. I have an article series that explains all of this and provides some plumbing to make it easy to use.

The Hyper-V Heartbeat Service

Due to isolation, the hypervisor can only directly sense if a virtual machine is in a powered on state. Beyond that, whatever happens inside the VM stays inside the VM. The heartbeat service is a simple sort of up/down tool. If the service can respond to the hypervisor, then the guest operating system is running at least well enough to start services. There may be some more detailed checks included, but that’s the basic summary of this service.

The Hyper-V Backup (volume shadow copy) Service

The volume shadow copy service (usually known as VSS) is a mechanism that Windows uses to facilitate crash- and application-consistent backups without stopping a computer. It functions by quiescing I/O and notifying applications that a backup will be occurring. Virtual machines present a problem if you want to back them up from the host. Even though Hyper-V can pause processing for a virtual machine, it can’t directly quiesce the guest operating system. So, its backup integration service operates as an intermediary.

When backup begins on the host, it can use this integration service to notify the VSS service within the guest. On Linux, the integration services uses a similar mechanism, although it can only quiesce I/O.

In older versions of Hyper-V, virtual machine backup used the standard VSS mechanism in the host. In modern versions, it relies on Hyper-V’s checkpointing mechanism instead. However, in-guest operations are still controlled by VSS.

Hyper-V Guest Services

I really wish that Microsoft had come up with a better name than just Guest services. Without context, it’s tough to realize that this is not a synonym for the overall category of Hyper-V integration services. The Guest services Hyper-V service allows you to copy files directly between the host and the guest.

How to Manipulate Integration Services in PowerShell

You should not manipulate services from within the guest. Use PowerShell or the GUI at the Hyper-V level. It will automatically handle service status within the guest.

Be aware that you can’t force a feature to work if the guest’s integration services don’t support it. For instance, you can’t enable the Guest Service for a Windows 2008 R2 guest and then copy a file into it.

Retrieve Integration Services in PowerShell

To see the guest services and their statuses for a virtual machine, use Get-VMIntegrationService.

Get-VMIntegrationService -VMName svdc1


From the output, you can see if a service is enabled and, if it is, its current status. Note that disabled services will always show a primary status of OK. This particular VM shows a nice example for secondary statuses as well. For the Heartbeat service, you can see that its monitored indicators are all OK. For VSS, it wants to tell us something about the component version, but we can’t read it all here.

To expand a secondary status description, specify the Name of the integration service to check, enclose the whole thing in parenthesis, and use the dot selector to pick the SecondaryStatusDescription property. Ex:
(Get-VMIntegrationService -VMName svdc1 -Name ‘VSS’).SecondaryStatusDescription.


Basically, the integration service is out of date, so I might have problems backing this one up.

Enable Integration Services in PowerShell

Enable an integration service with Enable-VMIntegrationService.

Enable-VMIntegrationService -VMName svdc1 -Name ‘Guest Service Interface’.

It will help if you run Get-VMIntegrationService first so that you can copy/paste the name. You can also use the pipeline to send objects from Get-VMIntegrationService to Enable-VMIntegrationService.

Disable Integration Services in PowerShell

Disable an integration service with Disable-VMIntegrationService.

Disable-VMIntegrationService -VMName svdc1 -Name ‘Guest Service Interface’.

It will help if you run Get-VMIntegrationService first so that you can copy/paste the name. You can also use the pipeline to send objects from Get-VMIntegrationService to Disable-VMIntegrationService.

How to Manipulate Integration Services in Hyper-V Manager

In Hyper-V Manager, you can determine if any given service is enabled. It will show the status of the Heartbeat service, but no others. Failover Cluster Manager uses the same dialog to show enabled/disabled state for services but does not show status for any of them.

On the virtual machine’s Settings dialog, look on the Integration Services tab. A service is enabled if it is checked. Be aware that checking a box for a service that the client does not support has no effect. For instance, you can check the Guest service box for a 2008 R2 guest, but you will not be able to copy files to/from it.

Integration Services for Hyper-V 2016 Management

To check the Heartbeat status, highlight the VM in Hyper-V Manager’s Virtual Machines pane. In the lower section, ensure that you are on the Summary tab. Look for the Heartbeat field:

Hyper-V Manager 2016

Leveraging Integration Services in Automation

I think that you can easily determine uses for these services, so I won’t throw a lot of silly ideas at you.

I would recommend that you find a way to incorporate regular checks of the Heartbeat service into your monitoring system. If the secondary status doesn’t return OK, there’s a solid chance that the virtual machine has crashed.

Look around a bit, and you might find other uses.

Have any questions or feedback?

Leave a comment below!

Hyper-V 2016 Host Mode: GUI vs Core

26 Sep 2017 by Eric Siron
Hyper-V Articles

Choice is a good thing, right? Well… usually. Sometimes, choice is just confusing. With most hypervisors, you get what you get. With Hyper-V, you can install in three different ways, and that’s just for the server hypervisor. In this article, we’ll balance the pros and cons of your options with the 2016 SKUs.

Server Deployment Options for Hyper-V

As of today, you can deploy Hyper-V in one of four packages.

Nano Server

When 2016 initially released, it brought a completely new install mode called “Nano”. Nano is little more than the Windows Server kernel with a tiny handful of interface bits attached. You then plug in the roles and features that you need to get to the server deployment that you want. I was not ever particularly fond of the idea of Hyper-V on Nano for several reasons, but none of them matter now. Nano Server is no longer supported as a Hyper-V host. It currently works, but that capability will be removed in the next iteration. Part of the fine print about Nano that no one reads includes the requirement that you keep within a few updates of current. So, you will be able to run Hyper-V on Nano for a while, but not forever.

If you currently use Nano for Hyper-V, I would start plotting a migration strategy now. If you are considering Nano for Hyper-V, stop.

Hyper-V Server

Hyper-V Server is the product name given to the free distribution vehicle for Hyper-V. You’ll commonly hear it referred to as “Hyper-V Core”, although that designation is both confusing and incorrect. You can download Hyper-V Server as a so-called “evaluation”, but it never expires.

A word of advice: Hyper-V Server includes a legally-binding license agreement. Violation of that licensing agreement subjects you to the same legal penalties that you would face for violating the license agreement of a paid operating system. Hyper-V Server’s license clearly dictates that it can only be used to host and maintain virtual machines. You cannot use it as a file server or a web server or anything else. Something that I need to make extremely clear: the license agreement does not provide special allowances for a test environment. I know of a couple of blog articles that guide you to doing things under the guise of “test environment”. That’s not OK. If it’s not legal in a production environment, it doesn’t magically become legal in a test environment.

Windows Server Core

When you boot to the Windows Server install media, the first listed option includes “Core” in the name. That’s not an accident; Microsoft wants you to use Core mode by default. Windows Server Core excludes the primary Windows graphical interface and explorer.exe. Some people erroneously believe that means that no graphical applications can be run at all. Applications that use the Explorer rendering engine will not function (such as MMC), but the base Windows Forms libraries and mechanisms exist.

Windows Server with GUI

I doubt that the GUI mode of Windows Server needs much explanation. You have the same basic graphical interface as Windows 10 with some modifications that make it more appropriate for a server environment. When you install from 2016 media, you will see this listed as (Desktop Experience).

The Pros and Cons of the Command-line and Graphical Modes for Hyper-V

I know that things would be easier if I would just tell you what to do. If I knew you and knew your environment, I might do that. I prefer giving you the tools and knowledge to make decisions like this on your own, though. So, we’ll complement our discussion with a pros and cons list of each option. After the lists, I’ll cover some additional guidelines and points to consider.

Hyper-V Server Pros and Cons

If you skipped the preamble, remember that “Hyper-V Server” refers to the completely free SKU that you can download at any time.

Pros of Hyper-V Server:

  • Never requires a licensing fee
  • Never requires activation
  • Smallest deployment
  • Smallest “surface area” for attacks
  • Least memory usage by the management operating system
  • Fewest patch needs
  • Includes all essential features for running Hyper-V (present, not necessarily enabled by default):
    • Hyper-V hypervisor
    • Hyper-V PowerShell interface
    • Cluster membership
    • Domain membership
    • Hyper-V Replica membership
    • Remote Desktop Virtual Host role for VDI deployments
    • RemoteFx (automatic with RDVH role)

Cons of Hyper-V Server:

  • Cannot provide Automatic Virtual Machine Activation
  • Cannot provide deduplication features
  • Impossible to enable the Windows Server GUI
  • Software manufacturers may refuse to support their software on it
  • Third-party support operations, such as independent consulting firms, may not have any experience with it
  • Switching to Windows Server requires a complete reinstall
  • Difficult to manage hardware

Hyper-V in Windows Server Core Pros and Cons

If you’ve seen the term “Hyper-V Core”, that probably means “Hyper-V Server”. This section covers the authentic Windows Server product installed in Core mode.

Pros of Windows Server Core for Hyper-V:

  • Microsoft recommends Windows Server Core for Hyper-V
  • Receives feature updates on the quickest schedule (look toward the bottom of the link in the preceding bullet)
  • Comparable deployment size to Hyper-V Server
  • Comparable surface area to Hyper-V Server
  • Comparable memory usage to Hyper-V Server
  • Comparable patch requirements to Hyper-V Server
  • Allows almost all roles and features of Windows Server
  • Can provide Automatic Activation for Windows Server in VMs (Datacenter Edition only)

Cons of Windows Server Core for Hyper-V:

  • Impossible to enable the Windows Server GUI
  • Must be licensed and activated
  • Upgrading to the next version requires paying for that version’s license, even if you will wait to deploy newer guests
  • Software manufacturers may refuse to support their software on it
  • Third-party support operations, such as independent consulting firms, may not have any experience with it
  • Difficult to manage hardware

Hyper-V in Windows Server GUI Pros and Cons

We saved what many consider the “default” option for last.

Pros of Windows Server with GUI for Hyper-V:

  • Familiar Windows GUI
  • More tools available, both native and third party
  • Widest support from software manufacturers and consultants
  • Easiest hardware management
  • Valid environment for all Windows Server roles, features, and software
  • Can provide Automatic Activation for Windows Server in VMs (Datacenter Edition only)

Cons of Windows Server with GUI for Hyper-V:

  • Familiarity breeds contempt
  • Slowest feature roll-out cycle (see the bottom of this article)
  • Largest attack surface, especially with explorer.exe
  • Largest deployment size
  • Largest memory usage
  • Largest patch requirements
  • Must be licensed and activated
  • Upgrading to the next version requires paying for that version’s license, even if you will wait to deploy newer guests

Side-by-Side Comparison of Server Modes for Hyper-V

Two items appear in every discussion of this topic: disk space and memory usage. I thought that it might be enlightening to see the real numbers. So, I built three virtual machines running Hyper-V in nested mode. The first contains Hyper-V Server, the second contains Windows Server Datacenter Edition in Core mode, and the third contains Windows Server Datacenter Edition in GUI mode. I have enabled Hyper-V in each of the Windows Server systems and included all management tools and subfeatures (
Add-WindowsFeature -Name Hyper-V -IncludeAllSubFeature -IncludeManagementTools). All came from the latest MSDN ISOs. None are patched. None are on the network.

Disk Usage Comparison of the Three Modes

I used the following PowerShell command to determine the used space:
‘{0:N0}’ -f (Get-WmiObject -Class Win32_LogicalDisk | ? DeviceId -eq ‘C:’ | % {$_.Size $_.FreeSpace}).

Deployment Mode Used Disk Space (bytes)
Hyper-V Server 2016 6,044,270,592
Windows Server 2016 Datacenter Edition in Core mode 7,355,858,944
Windows Server 2016 Datacenter Edition in GUI mode 10,766,614,528

For shock value, the full GUI mode of Windows Server adds 78% space utilization above Hyper-V Server 2016 and 46% space utilization above Core mode. That additional space amounts to less than 5 gigabytes. If 5 gigabytes will make or break your deployment, you’ve got other issues.

Memory Usage Comparison of the Three Modes

We’ll start with Task Manager while logged on:


These show what we expect: Hyper-V Server uses the least amount of memory, Windows Server Core uses a bit more, and Windows Server with GUI uses a few ticks above both. However, I need to point out that these charts show a more dramatic difference than you should encounter in reality. Since I’m using nested VMs to host my sample systems, I only gave them 2 GB total memory apiece. The consumed memory distance between Hyper-V Server and Windows Server with GUI weighs in at a whopping .3 gigabytes. If that number means a lot to you in your production systems, then you’re going to have other problems.

But that’s not the whole story.

Those numbers were taken from Task Manager while logged on to the systems. Good administrators log off of servers as soon as possible. What happens, then, when we log off? To test that, I had to connect each VM to the network and join the domain. I then ran:
Get-WmiObject Win32_OperatingSystem | select FreePhysicalMemory with the ComputerName switch against each of the hosts. Check out the results:

Deployment Mode Free Memory (MB)
Hyper-V Server 2016 1,621,148
Windows Server 2016 Datacenter Edition in Core mode 1,643,060
Windows Server 2016 Datacenter Edition in GUI mode 1,558,744

Those differences aren’t so dramatic, are they? Windows Server Core even has a fair bit more free memory than Hyper-V Server… at that exact moment in time. If you don’t have much background in memory management, especially in terms of operating systems, then keep in mind that memory allocation and usage can seem very strange.

The takeaway: memory usage between all three modes is comparable when they are logged off.

Hyper-V and the “Surface Area” Argument

Look at the difference in consumed disk sizes between the three modes. Those extra bits represent additional available functionality. Within them, you’ll find things such as Active Directory Domain Services and IIS. So, when we talk about choosing between these modes, we commonly point out that all of these things add to the “attack surface”. We try to draw the conclusion that using a GUI-less system increases security.

First part: Let’s say that a chunk of malware injects itself into one of the ADDS DLLs sitting on your Windows Server host running Hyper-V. What happens if you never enable ADDS on that system? Well, it’s infected, to be sure. But, in order for any piece of malware to cause any harm, something eventually needs to bring it into memory and execute it. But, you know that you’re not supposed to run ADDS on a Hyper-V host. Philosophical question: if malware attacks a file and no one ever loads it, is the system still infected? Hopefully, you’ve got a decent antimalware system that will eventually catch and clean it, so you should be perfectly fine.

On one hand, I don’t want to downplay malware. I would never be comfortable with any level of infection on any system. On the other hand, I think common sense host management drastically mitigates any concerns. I don’t believe this is enough of a problem to carry a meaningful amount of weight in your decision.

Second part: Windows Server runs explorer.exe as its shell and includes Internet Explorer. Attackers love those targets. You can minimize your exposure by, you know, not browsing the Internet from a server, but you can’t realistically avoid using explorer.exe on a GUI system. However, as an infrastructure system, you should be able to safely instruct your antimalware system to keep a very close eye on Explorer’s behavior and practice solid defensive techniques to prevent malware from reaching the system.

Overall takeway from this section: Explorer presents the greatest risk. Choose the defense-in-depth approach of using Hyper-V Server or Windows Server Core, or choose to depend on antimalware and safe operating techniques with the Windows Server GUI.

Hyper-V and the Patch Frequency Non-Issue

Another thing that we always try to bring into these discussions is the effect of monthly patch cycles. Windows Server has more going on than Hyper-V Server, so it gets more patches. From there, we often make the argument that more patches equals more reboots.

A little problem, though. Let’s say that Microsoft releases twelve patches for Windows Server and only two apply to Hyper-V Server. One of those two patches requires a reboot. In that case, both servers will reboot. One time. So, if we get hung up on downtime over patches, then we gain nothing. I believe that, in previous versions, the downtime math did favor Hyper-V Server a few times. However, patches are now delivered in only a few omnibus packages instead of smaller targeted patches. So, I suspect that we will no longer be able to even talk about reboot frequency.

One part of the patching argument remains: with less to patch, fewer things can go wrong from a bad patch. However, this argument faces the same problem as the “surface area” non-issue. What are you using on your Windows Server system that you wouldn’t also use on a Hyper-V Server system? If you’re using your Windows Server deployment correctly, then your patch risks should be roughly identical.

Most small businesses will patch their Hyper-V systems via automated processes that occur when no one is around. Larger businesses will cluster Hyper-V hosts and allow Cluster Aware Updating to prevent downtime.

Overall takeaway from this section: patching does not make a convincing argument in any direction.

Discussions: Choosing Between Core and GUI for Hyper-V

Now you’ve seen the facts. You’ve seen a few generic arguments for the impact level of two of those facts. If you still don’t know what to do, that’s OK. Let’s look at some situational points.

A Clear Case for Hyper-V on Windows Server Full GUI

If you’re in a small environment with only a single physical server, go ahead and use the full GUI.

Why? Some reasons:

  • It is not feasible to manage Hyper-V without any GUI at all. I advocate for PowerShell usage as strongly as anyone else, but sometimes the GUI is a better choice. In a multi-server environment, you can easily make a GUI-less system work because you have at least one GUI-based management system somewhere. Without that, GUI-less demands too much.
  • The world has a shortage of Windows Server administrators that are willing and able to manage a GUI-less system. You will have difficulty hiring competent help at a palatable price.
  • Such a small shop will not face the density problems that justify the few extra resources saved by the GUI-less modes.
  • The other issues that I mentioned are typically easier to manage in a small environment than in a large environment.
  • A GUI system will lag behind Core in features, but Hyper-V is quite feature-complete for smaller businesses. You probably won’t miss anything that really matters to you.
  • If you try Hyper-V Server or Windows Server Core and decide that you made a mistake, you have no choice but to reinstall. If you install the GUI and don’t want to use it, then don’t use it — switch to remote management practices. You won’t miss out on anything besides the faster feature release cycle.

We can make some very good arguments for a GUI-less system, but none are strong enough to cause crippling pain for a small business. When the GUI fits, use it.

A Clear Case for Hyper-V Server

Let’s switch gears completely. Let’s say that:

  • You’re a PowerShell whiz
  • You’re a cmd whiz
  • You run a lot of Linux servers
  • Your Windows Servers (if any) are all temporary testing systems

Hyper-V Server will suit you quite well.

Everyone Else

If you’re somewhere in the middle of the above two cases, I think that Microsoft’s recommendation of Windows Server Core with Hyper-V fits perfectly. The parts that stand out to me:

  • Flexibility: Deduplication has done such wonders for me in VDI that I’m anxious to see how I can apply it to server loads. In 2012 R2, server guests were specifically excluded; VDI only. Server 2016 maintains the same wording in the feature setup, but I can’t find a comparable statement saying that server usage is verboten in 2016. I could also see a case for building a nice VM management system in ASP.Net and hosting it locally with IIS — you can’t do that in Hyper-V Server.
  • Automatic Virtual Machine Activation. Who loves activation? Nobody loves activation! Let the system deal with that.
  • Security by terror: Not all server admins are created equally. I find that the really incompetent ones won’t even log on to a Server Core/Hyper-V Server system. That means that they won’t put them at risk.
  • Remote management should be the default behavior. If you don’t currently practice remote management, there’s no time like the present! You can dramatically reduce the security risk to any system by never logging on to its console, even by RDP.

You can manage Hyper-V systems from a Windows 10 desktop with RSAT. It’s not entirely without pain, though:

  • Drivers! Ouch! Microsoft could help us out by providing a good way to use Device Manager remotely. We should not let driver manufacturers off the hook easily, though. Note: Honolulu is coming to reduce some of that pain.
  • Not everyone that requires the GUI is an idiot. Some of them just haven’t yet learned. Some have learned their way around PowerShell but don’t know how to use it for Hyper-V. You like taking vacations sometimes, don’t you?
  • Crisis mode when you don’t know what’s wrong can be a challenge. It’s one thing to keep the top spinning; it’s another to get it going when you can’t see what’s holding it down. However, these problems have solutions. It’s a pain, but a manageable one.

I’m not here to make the decision for you. You now have enough information to make an informed decision.

Have any questions or feedback?

Leave a comment below!

Docker’s routing mesh available with Windows Server version 1709

The Windows Core Networking team, along with our friends at Docker, are thrilled to announce that support for Docker’s ingress routing mesh will be supported with Windows Server version 1709.

Ingress routing mesh is part of swarm mode–Docker’s built-in orchestration solution for containers. Swarm mode first became available on Windows early this year, along with support for the Windows overlay network driver. With swarm mode, users have the ability to create container services and deploy them to a cluster of container hosts. With this, of course, also comes the ability to define published ports for services, so that the apps that those services are running can be accessed by endpoints outside of the swarm cluster (for example, a user might want to access a containerized web service via web browser from their laptop or phone).

To place routing mesh in context, it’s useful to understand that Docker currently provides it, along with another option for publishing services with swarm mode–host mode service publishing:*

  • Host mode is an approach to service publishing that’s optimal for production environments, where system administrators value maximum performance and full control over their container network configuration. With host mode, each container of a service is published directly to the host where it is running.
  • Routing mesh is an approach to service publishing that’s optimized for the developer experience, or for production cases where a simple configuration experience is valued above performance, or control over how incoming requests are routed to the specific replicas/containers for a service. With ingress routing mesh, the containers for a published service, can all be accessed through a single “swarm port”–one port, published on every swarm host (even the hosts where no container for the service is currently running!).

While our support for routing mesh is new with Windows Server version 1709, host mode service publishing has been supported since swarm mode was originally made available on Windows. 

*For more information, on how host mode and routing mesh work, visit Docker’s documentation on routing mesh and publishing services with swarm mode.

So, what does it take to use routing mesh on Windows? Routing mesh is Docker’s default service publishing option. It has always been the default behavior on Linux, and now it’s also supported as the default on Windows! This means that all you need to do to use routing mesh, is create your services using the --publish flag to the docker service create option, as described in Docker’s documentation.

For example, assume you have a basic web service, defined by a container image called, web-frontend. If you wanted to publish this service to port 80 of each container and port 8080 of all of your swarm nodes, you’d create the service with a command like this:

C:> docker service create --name web --replicas 3 --publish 8080:80 web-frontend

In this case, the web app, running on a pre-configured swarm cluster along with a db backend service, might look like the app depicted below. As shown, because of routing mesh clients outside of the swarm cluster (in this example, web browsers) are able to access the web service via its published port–8080. And in fact, each client can access the web service via its published port on any swarm host; no matter which host receives an original incoming request, that host will use routing mesh to route the request to a web container instance that can ultimately service that request.

Once again, we at Microsoft and our partners at Docker are proud to make ingress mode available to you on Windows. Try it out on Windows Server version 1709, and using Docker EE Preview*, and let us know what you think! We appreciate your engagement and support in making features like routing mesh possible, and we encourage you to continue reaching out with feedback. Please provide your questions/comments/feature requests by posting issues to the Docker for Windows GitHub repo or by emailing the Windows Core Networking team directly, at

*Note: Ingress mode on Windows currently has the following system requirements:

Microsoft Ignite 2017 – Latest News, Updates and Rumors

22 Sep 2017 by Andy Syrewicze
Altaro News

It’s back! Next week marks one of my favorite times of the year. Yes, this coming week brings us Microsoft Ignite 2017, where IT Pros, Speakers, and MVPs from around the world converge in one location to talk about the latest innovations in the Microsoft Technology Stack, swap war stories and help one another solve technology problems. I’ve always been amazed at the sense of community, not only from attendees but from Microsoft itself during MS Ignite. There really is an immense feeling of community at this event every year, which is probably the thing I look forward to the most. Previous years have seen 10k to 15k+ in attendance at this event quite consistently and as such, there is a lot of knowledge to go around. This is often in the form of community theater sessions, in-person immersion experiences, and Microsoft-led speaking slots.

1 – Microsoft Ignite 2016 in Atlanta via Andy Syrewicze

Taking it all in and digesting that information can be difficult, but this year I aim to be of assistance to our readers and followers on social media who want to get the most they can out of this event (even if they can’t attend in person). To do this to the best of my ability, I’m ditching the booth at this event as a vendor to take up the role as a fellow attendee in the trenches. Not only do I get to experience it as you do, I’m also aiming to bring a little bit of an extra touch to your experience through live video feeds, and interviews with distinguished industry experts, Microsoft MVPs, and interesting personalities!

The growing list of confirmed interviews is below!

When is Microsoft Ignite 2017?

Officially, Microsoft Ignite runs from September 25th to the 29th. However, Sunday is full of pre-day events, and I will be filming a number of interviews that day, so you can be sure we’ll have some fresh video content for you starting aready from Sunday night and into Monday morning!

Where is Ignite this year?

Ignite is being hosted in sunny Orlando Florida this year! The venue should lend itself well for a conference of this size, and the closing celebration on Thursday night will be hosted at Universal Studios, so I’ll be sure to post some pictures and live video of the proceedings so you can join in the fun and discussion.

What Kind of Content Will You be Providing?

Well, that’s really part of the fun right? Would it be as fun if you knew exactly what’s in store? Ok, if you MUST know, we’ll be sending a number of broadcasts directly from the show to discuss some of the new reveals and some of the new features that are sure to make an impact in your day-to-day operations. Additionally, we’ll be getting some of this information directly from leaders in the industry.

Here’s a taste of the interviews coming your way:

Rick Claus – Sr. Program Manager – Azure Compute Team

Kay Williams – Principal PM Manager at Microsoft

Michael Bender – IT Instructor at Madison College and Founder of #TheKrewe

Dave Kawula – Microsoft MVP and Enterprise Consultant and co-author of the eBook 6 Hyper-V Configuration Mistakes to Avoid

Thomas Maurer – Microsoft MVP and Cloud Architect and past guest on the Altaro webinar Future-proofing your datacenter with Microsoft Azure Stack

This is just a fraction of the content we’re providing this year, but you get the idea! We’ll be sure to keep you updated via social channels as things progress throughout the week. Speaking of social channels…..

How Do I Follow Along?

We’ll be doing the bulk of the information sharing and notifications via our various social platforms (below). Keep an eye on these, and be sure to keep an eye right here on our Hyper-V Blog after the event for a summarizing post containing all the content and topic discussions from the week of Ignite!

Twitter: @AltaroSoftware and @asyrewicze

Facebook: Altaro Software

LinkedIn: Altaro Software and Andy Syrewicze

Google+: Altaro Software

Tell me what you want to know!

Let me know if the comments section or directly over the social channels above. If there is a certain type of content you’re interested in or a particular speaker you’d like to see an interview with, let me know in the comments section below and I’ll be sure to try and accommodate you.

Also, if you’re going to be at the show and you’d like to talk in person, be sure to let me know below as well. I’m looking forward to meeting as many people as possible, just send me a message – I’m always open to a discussion about technology over a cold beverage!

We hope to make it a great #MSIgnite, and we hope to see you there (in person, or virtually)!

Have any questions or feedback?

Leave a comment below!

Project ‘Honolulu’: What you need to know

19 Sep 2017 by Eric Siron
Windows Server

The biggest problem with Hyper-V isn’t Hyper-V at all. It’s the management experience. We’ve all had our complaints about that, so I don’t think a rehash is necessary. Thing is, Hyper-V is far from alone. Microsoft has plenty of management issues across its other infrastructure roles and features as well. Enter Project ‘Honolulu’: an attempt to unify and improve the management experience for Microsoft’s infrastructure offerings.

Before I get very far into this, I want one thing to be made abundantly clear: the Honolulu Project is barely out of its infancy. As I write this, it is exiting private preview. The public beta bits aren’t even published yet.

With that said, unless many things change dramatically between now and release, this is not the Hyper-V management solution that you have been waiting for. At its best, it has a couple of nice touches. In a few cases, it is roughly equivalent to what we have now. For most things, it is worse than what we have available today. I hate to be so blunt about it because I believe that Microsoft has put a great deal of effort into Honolulu. However, I also feel like they haven’t been paying much attention to the complaints and suggestions the community has made regarding the awful state of Hyper-V management tools.

What is Project ‘Honolulu’

When you look at Honolulu, it will appear something like an Azure-ified Server Manager. It adopts the right-to-left layouts that the Azure tools use, as opposed to the up-and-down scrolling that we humans and our mice are accustomed to.

Thou shalt not use Honolulu in a window
Thou shalt not use Honolulu in a window

This sort of thing is normative for the Azure tools. If you have a 50″ 4k screen and nothing else to look at, I’m sure that it looks wonderful. If you are using VMConnect or one of those lower resolution slide-out monitors that are still common in datacenters, then you might not enjoy the experience. And yes, the “<” icon next to Tools means that you can collapse that panel entirely. It doesn’t help much. I don’t know when it became passé for columns to be resizable and removable. Columns should be resizable and removable.

As you see it in that screenshot, Honolulu is running locally. It can also run in a gateway mode on a server. You can then access it from a web browser from other systems and devices.

Requirements for Running Project ‘Honolulu’

For the Honolulu Project itself, you can install on:

  • Windows 10
  • Windows Server 2012 through 2016

On a Windows 10 desktop or a Server 2012 system, it will only be accessible locally.

If you install on a server 2012 R2 through 2016 SKU, it will operate in the aforementioned gateway mode. You just open a web browser to that system on whatever port you configure, ex: https://managementmentsystem:6516. You will be prompted for credentials.

When you provide credentials to Honolulu, the systems that you connect to will be associated with your account. If you connect to Honolulu with a different user account, it will not display any of the servers that were chosen under a different account. Each need to be set up separately. You can import lists to reduce the pain.

Note: As it stands right now, I cannot get Honolulu work on a 2012 R2 system. It will open, but then refuses to connect to any server in my organization. I am actively working on this problem and will report back if a solution can be found. That’s one of the dangers of using early software, not a lifelong condemnation of the product.

Requirements for Targets of Honolulu

The target system(s) must be a Server SKU 2012 through 2016. It/they must have Windows Management Framework 5 or higher loaded. The easiest way to tell is by opening a PowerShell prompt and running
$PSVersionTable. The PowerShell version and the Windows Management Framework version will always be the same. It also helps if you can verify that you can connect from the management system to the target with

The following screenshot shows an example. I first tested that my management system has the correct version. Then I connected to my target and checked the WMF version there. I should have no problems setting up the first system to run Project Honolulu to connect to the second system.

running project honolulu

If you are running all of the systems in the same domain, then this will all “just work”. I’m not sure yet how cross-domain authentication works. If you’ve decided that security is unimportant and you’re running your Hyper-V host(s) in workgroup mode, then you will need to swing the door wide open to attackers by configuring TrustedHosts on the target system(s) to trust any computer that claims to have the name of your Honolulu system.

Requirements for Viewing Project ‘Honolulu’

Honolulu presents its views via HTML 5 web pages. Edge and Chrome work well. Internet Explorer doesn’t work at all:


I think it will be interesting to see how that plays out in the enterprise. Windows 10 isn’t exactly the best corporate player, so several organizations are hanging on to Windows 7. Others are moving to Windows 10, but opting for the Long-Term Servicing Branch (LTSB). LTSB doesn’t include Edge. So, is Microsoft (inadvertently?) pushing people toward Google Chrome?

Connecting to a Target Server in Honolulu

When you first start up Honolulu, you have little to look at:


Click the + Add link to get started adding systems. Warning: If you’re going to add clusters, do that following the instructions in the next section. Only follow this for stand-alone hosts.

Type the name of a system to connect to, and it will automatically start searching. Hopefully, it will find the target. You can click the Submit button whether it can find it or not.

A working system:


A non-working system:


As you can see in the links, you can also Import Servers. For this, you need to supply a text file that contains a list of target servers.

Connecting to a Target Cluster in Honolulu

Honolulu starts out in “Server Manager” mode, so it will only connect to servers. If you try to connect it to a failover cluster in Server Manager mode, it will pick up the owning node instead. In order to connect to a failover cluster, you need to switch the mode.

At the top of the window, find the Server Manager heading. Drop that down and select Failover Cluster Manager.


Now, add clusters with the + Add button. When it detects the cluster, it will also prompt you to add the nodes as members of Server Manager:


Windows Management Framework Error for Honolulu

As mentioned in the beginning, every target system needs to have at least Windows Management Framework version 5 installed. If a target system does not meet that requirement, Honolulu will display that status:


The Really Quick Tour for Honolulu

I focus on Hyper-V and I’m certain that dozens of other Honolulu articles are already published (if not more). So, let’s burn through the non-Hyper-V stuff really fast.

Right-click doesn’t do anything useful anywhere in Honolulu. Train yourself to use only the left mouse button.

Server Manager has these sections:

  • Overview: Shows many of the things that you can see in Computer Properties. Also has several real-time performance charts, such as CPU and memory. For 2016+ you can see disk statistics. I like this page in theory, but the execution is awful. It assumes that you always want to see the basic facts about a host no matter what and that you have a gigantic screen resolution. My VMConnect screen is set to 1366×768 and I can’t even see a single performance chart in its entirety:
  • Certificates: No more dealing with all the drama of manually adding the certificates snap-in! Also, you can view the computer and user certificates at the same time! Unfortunately, it doesn’t look like you can request a new certificate, but most other functionality seems to be here.
  • Devices: You can now finally see the devices installed on a Server Core/Hyper-V Server installation. You can’t take any action except Disable, unfortunately. It’s still better than what we had.
  • Events: Event Viewer, basically.
  • Files: Mini-File Explorer in your browser! You can browse the directory structure and upload/download files. You can view properties, but you can’t do anything with shares or permissions.
  • Firewall: Covers the most vital parts of firewall settings (profile en/disabling and rule definitions).
  • Local Users and Groups: Add and remove local user accounts. Add them to or remove them from groups. You cannot add or delete local groups. Adding a user to a group is completely free-text; no browsing. Also, if you attempt to add a user that doesn’t exist, you get a confirmation message that tells you that it worked, but the field doesn’t populate.
  • Network: View the network connections and set basic options for IPv4 and IPv6.
  • Processes: Mostly like Task Manager. Has an option to Create Process Dump.
  • Registry: Nifty registry editor; includes Export and Import functions. Very slow, though; personally I’d probably give up and use regedit.exe for as long as I’m given a choice.
  • Roles and Features: Mostly what you expect. No option for alternate install sources, though, so you won’t be using it to install .Net 3.5. Also, I can’t tell how to discard accidental changes. No big deal if you only accidentally checked a single item. For some reason, clicking anywhere on a line toggles the checked/not checked state, so you can easily change something without realizing that you did it.
  • Services: Interface for installed services. Does not grant access to any advanced settings for a service (like the extra tabs on the SNMP Service). Also does not recognize the Delayed Start modifier for Automatic services. I would take care to only use this for Start and Stop functions.
  • Storage: Works like the Storage part of the Files and Storage Services section in Server Manager. Like the preceding sections, includes most of the same features as its real Server Manager counterpart, but not all.
  • Storage Replica: I’m not using Storage Replica anywhere so I couldn’t gauge this one. Requires a special setup.
  • Virtual Machines and Virtual Switches: These two sections will get more explanation later.
  • Windows Update: Another self-explanatory section. This one has most of the same functionality as its desktop counterpart, although it has major usability issues on smaller screens. The update list is forced to yield space to the restart scheduler, which consumes far more screen real estate than it needs to do its job.

Virtual Switches in Honolulu

Alphabetically, this comes after Virtual Machines, but I want to get it out of the way first.

The Virtual Switches section in Project ‘Honolulu’ mostly mimics the virtual switch interface in Hyper-V Manager. So, it gets props for being familiar. It takes major dings for duplicating Hyper-V Manager’s bad habits.

First, the view:



  • New Virtual Switch
  • Delete Virtual Switch
  • Rename Virtual Switch
  • Modify some settings of a virtual switch

The Settings page (which I had to stitch together because it successfully achieves the overall goal of wasting maximal space):


The New Virtual Switch screen looks almost identical, except that it’s in a sidebar so it’s not quite as wide.

Notes on Honolulu’s virtual switch page:

  • Copies Hyper-V Manager’s usage of the adapter’s cryptic Description field instead of its name field.
  • If you look in the Network Adapter setting on the Settings for vSwitch screenshot and then compare it to the overview screen shot, you should notice something: It didn’t pick the team adapter that I really have my vSwitch on. Also, you can’t choose the team adapter. I didn’t tinker with that because I didn’t want to break my otherwise functional system, but not being able to connect a virtual switch to a team is a non-starter for me.
  • Continues to use the incorrect and misleading “Share” terminology for “Shared with Management OS” and “Allow management OS to share this network adapter”. Hey Microsoft, how hard would it really be to modify those to say “Used by Management OS” and “Allow management OS to use this virtual switch”?
  • No VLAN settings.
  • No SR-IOV settings.
  • No Switch-Embedded Teaming settings
  • No options for controlling management OS virtual NICs beyond the first one

Virtual Machines in Honolulu

All right, this is why we’re here! Make sure that you’re over something soft or the let-down might sting.

Virtual Machine Overview

The overview is my favorite part, although it also manifests the wasteful space usage that plagues this entire tool. Even on a larger resolution, it’s poorly made. However, I like the information that it displays, even if you need to scroll a lot to see it all.

At the top, you get a quick VM count and a recap of recent events:


Even though I like the events being present, that tiny list will be mostly useless on an environment of any size. Also, it might cause undue alarm. For instance, those errors that you see mean that Dynamic Memory couldn’t expand any more because the VMs had reached their configured maximum. You can’t see that here because it needs two inches of whitespace padding to its left and right.

You can also see the Inventory link. We’ll come back to that after the host resources section.

Virtual Machine Host Resource Usage

I mostly like the resource view. Even on my 1366×768 VMConnect window, I have enough room to fit the CPU and memory charts side-by-side. But, they’re stacked and impossible to see together. I’ve stitched the display for you to see what it could look like with a lot of screen to throw at it:


Virtual Machine Inventory

Back at the top of the Virtual Machines page, you can find the Inventory link. That switches to a page where you can see all of the virtual machines:


That doesn’t look so bad, right? My primary complaint with the layout is that I believe that the VM’s name should be prioritized. I’d rather have an idea of the VM’s name as opposed to the Heart Beat or Protected statuses, if given a choice.

My next complaint is that, even at 1366×768, which is absolutely a widescreen resolution, the elements have some overrun. If I pick a VM that’s on, I must be very careful when trying to access the More menu so that I don’t inadvertently Shutdown the guest instead:


What’s on that More menu? Here you go:


That’s for a virtual machine that’s turned on. No, your eyes are not deceiving you. You cannot modify any of the settings of a virtual machine while it is running. Power states and checkpoints are the limit.

I don’t know what Protected means. It’s not about being shielded or clustered. I suppose it means that it’s being backed up to Azure? If you’re not using Azure backup then this field just wastes even more space.

Virtual Machine Settings

If you select a virtual machine that’s off, you can then modify its settings. I elected not to take all of those screenshots. Fitting with the general Honolulu motif, they waste a great deal of space and present less information than Hyper-V Manager. These setting groupings are available:

  • General: The VM’s name, notes, automatic start action, automatic stop action, and automatic critical state action
  • Memory: Startup amount, Dynamic Memory settings, buffer, and weight
  • Processors: Number only. No NUMA, compatibility mode, reservation, or weight settings
  • Disks: I could not get the disks tab to load for any virtual machine on any host, whether 2012 R2 or 2016. It just shows the loading animation
  • Networks: Virtual switch connection, VLAN, MAC (including spoofing), and QoS. Nothing about VMQ, IOV, IPSec, DHCP Guard, Router Guard, Protected Network, Mirroring, Guest Teaming, or Consistent Device Naming
  • Boot Order: I could not get this to load for any virtual machine.

Other Missing Hyper-V Functionality in Honolulu

A criticism that we often level at Hyper-V Manager is just how many settings it excludes. If we only start from there, Project ‘Honolulu’ excludes even more.

Features available in Hyper-V Manager that Honolulu does not expose:

  • Hyper-V host settings — any of them. Live Migration adapters, Enhanced Session Mode, RemoteFX GPUs, and default file locations
  • No virtual SAN manager. Personally, I can live with that, since people need to stop using pass-through disks anyway. But, there are some other uses for this feature and it still works, so it makes the list of Honolulu’s missing features.
  • Secure boot
  • VM Shielding
  • Virtual TPM
  • Virtual hardware add/remove
  • Indication of VM Generation
  • Indication/upgrade of VM version
  • Shared Nothing Live Migration (intra-cluster Live Migration does work; see the Failover Clustering section below)
  • Storage (Live) Migration
  • Hyper-V Replica
  • Smart page file

Except for the automatic critical action setting, I did not find anything in Project ‘Honolulu’ that isn’t in Hyper-V Manager. So, don’t look here for nested VM settings or anything like that.

Failover Clustering for Hyper-V in Honolulu

Honolulu’s Failover Cluster Manager is even more of a letdown than Hyper-V. Most of the familiar tabs are there, but it’s almost exclusively read-only. However, we Hyper-V administrators get the best of what it can offer.

If you look on the Roles tab, you can find the Move action. That initiates a Quick or Live Migration:


Unfortunately, it forces you to pick a destination host. In a small cluster like mine, no big deal. In a big cluster, you’d probably like the benefit of the automatic selector. You can’t even see what the other nodes’ load levels look like to help you to decide.

Other nice features missing from Honolulu’s Failover Cluster Manager:

  • Assignment, naming, and prioritizing of networks
  • Node manipulation (add/evict)
  • Disk manipulation (add/remove cluster disk, promote/demote Cluster Shared Volume, CSV ownership change)
  • Quorum configuration
  • Core resource failover
  • Cluster validation. The report is already in HTML, so even if this tool can’t run validation, it would be really nice if it could display the results of one

Showstopping Hyper-V Issues in Project ‘Honolulu’

Pay attention to the dating of this article, as all things can change. As of this writing, these items prevent me from recommending Honolulu:

  • No settings changes for running virtual machines. The Hyper-V team has worked very hard to allow us to change more and more things while the virtual machine is running. Honolulu negates all of that work, and more.
  • No Hyper-V switch on a team member
  • No VMConnect (console access). If you try to connect to a VM, it uses RDP. I use a fair number of Linux guests. Microsoft has worked hard to make it easy for me to use Linux guests. For Windows guests, RDP session cuts out the pre-boot portions that we sometimes need to see.
  • No host configuration

Any or all of these things might change between now and release. I’ll be keeping up with this project in hopes of being able to change my recommendation.

The Future of Honolulu

I need to stress, again, that Honolulu is just a baby. Yes, it needs a lot of work. My general take on it, though, is that it’s beginning life by following in the footsteps of the traditional Server Manager. The good: it tries to consolidate features into a single pane of glass. The bad: it doesn’t include enough. Sure, you can use Server Manager/Honolulu to touch all of your roles and features. You can’t use it as the sole interface to manage any of them, though. As-is, it’s a decent overview tool, but not much more.

Where Honolulu goes from here is in all of our hands. I’m writing this article a bit before the project goes into public beta, so you’re probably reading it at some point afterward. Get the bits, set it up, and submit your feedback. Be critical, but be nice. Designing a functional GUI is hard. Designing a great GUI is excruciatingly difficult. Don’t make it worse with cruel criticism.

Have any questions or feedback?

Leave a comment below!