Tag Archives: faster

Cloudian storage replaces tape at Vox Media

Needing more flexible and faster long-term storage than tape to deal with rapidly expanding data, Vox Media selected Cloudian storage to complement its Quantum StorNext primary storage.

Vox Media installed Cloudian object storage in early 2018 in its main New York office, with plans to expand it to other U.S. sites for disaster recovery.

Vox Media is a digital media company that runs online publications, including SB Nation, The Verge and Polygon. Its sites cater to an audience of tech, sports and video game enthusiasts. As the digital media landscape evolved, editorial content included more videos, streams and podcasts. With more than 1 PB of data, Vox Media’s on-site tape library was not archiving quickly enough to store the growing amount of rich media.

With Vox Media’s storage system filling up faster than it could be archived, it was vulnerable to data loss if there was a disaster. Sarah Semlear, director of post-production and technical operations at Vox Media, looked to object storage to patch that hole and discovered Cloudian.

Sarah Semlear, director of post-production and technical operations at Vox MediaSarah Semlear

Semlear first encountered Cloudian at a Sports Video Group event in 2017. Impressed with what she saw at the event, she then spoke with Chesapeake Systems, the company Vox Media uses for IT consultation and implementation. After seeing Cloudian in action and coming up with a proof of concept on how it would integrate with Vox Media’s existing SAN and media asset management system, Semlear purchased Cloudian storage.

Semlear looked at other object storage vendors, but said she selected Cloudian storage chiefly because it was the quickest and easiest to roll out. It also worked effectively with Vox’s installed infrastructure.

“A lot of the object storage was out of our price range or not completely set up yet,” she said. “We had talked to some other vendors that had made a lot of promises, but really hadn’t proven anything as far as capabilities go. They were saying, ‘Don’t worry about it. We’ll get it to you eventually.’ And we were like, ‘But we need it right now.'”

Semlear said she also found Cloudian storage easy to integrate with Vox Media’s infrastructure. Vox Media uses Quantum StorNext for its main storage system and Evolphin for its media asset management system, so Semlear needed a system that would play nicely with those products.

The amalgamation of different vendors that we’re working with helps us be flexible and helps us adapt to change.
Sarah Semleardirector of post-production and technical operations, Vox Media

“The nice thing about multiple vendors is that when you just have one system running everything, there’s less flexibility. The amalgamation of different vendors that we’re working with helps us be flexible and helps us adapt to change,” she said.

Ever since deploying Cloudian in early 2018, Semlear’s been impressed by its performance — especially relative to tape. Writing to the Cloudian archive and retrieving data from it took far less time than tape. Semlear stressed how important quick turnarounds were in the editorial world. “We live in a short-attention-span time period. Folks want their content right now,” she said.

With about 1.3 PB of data archived in a Cloudian storage appliance in Vox Media’s New York City office, Semlear plans to take advantage of Cloudian’s multisite feature eventually. Her plan is to install instances in the San Francisco and Washington, D.C., offices and have everything synced in an intelligent way. For example, she wants The Verge content to only live on the San Francisco and New York machines, while Vox site content should only be archived in New York and D.C.

As for the old LTO system, Vox Media is still pulling archived data off it, but slowly phasing it out entirely. “We’re not writing to tape anymore. We’re just writing to the Cloudian archive,” Semlear said. In the meantime, she said she’s searching for a longer-term cold storage option to which the existing Cloudian storage can push older content.

Vexata fires up NVMe flash storage to tackle analytics

NVMe flash storage vendor Vexata has rewired its ignition to enable faster cognition of dense analytics.

The startup last week upgraded its VX-OS operating system for capacity and performance. VX-OS 3.5 allows customers to expand a VX array nondisruptively from 180 TB to nearly half a petabyte in 6U.

Vexata is among a handful of startups with NVMe flash array storage systems designed from scratch for nonvolatile memory express flash technologies. Its cut-through architecture offloads the data path on field programmable gate arrays. Vexata arrays store metadata in DRAM, and Vexata’s internal Ethernet fabric parallelizes all data in the system for distributed shared storage. 

NVMe is viewed as a faster alternative to the SCSI protocol. Most established array vendors allow customers to replace SAS and SATA SSDs with NVMe flash drives, although the market for rack-scale flash systems is evolving.

The next phase of development is a fully developed ecosystem for the NVMe over Fabrics (NVMe-oF) specification. NVMe-oF enables NVMe message commands to be transferred between a host server and storage array via Ethernet, Fibre Channel or InfiniBand. Today, PCIe-connected NVMe drives require Remote Direct Memory Access for data transfer.

“We are targeting this (product) to cognitive computing, where we increasingly are finding a number of large opportunities. Customers want to talk about AI, machine learning

and
autonomous systems. These aren’t fringe applications. Organizations are mainstreaming this stuff,” Vexata CEO Zahid Hussain said.

VX-OS 3.5 was engineered for improved latency on mixed read-write random workloads. Hussain said Vexata storage now can ingest writes at about 1.5 TB per minute, bringing it closer in line with its 3 TB to 4 TB per minute ingest rate on reads.

Most vendors design their all-flash and hybrid arrays with a dedicated clutch of SSDs to cache hot data, plus either disk or flash as back-end storage. Caching speeds performance, but it also limits the size of the working data set. 

Hussain said using up all the NVMe SSDs provides storage for denser computations and can modify data as it is being read.

“The strategy before was to add more controllers, but that would only activate a smaller number of the total available SSDs in the array. We have GPU-

ified
the whole architecture to solve that problem,” Hussain said.

VX-100 NVMe blades double up on capacity

The first iteration of Vexata’s array hardware scaled capacity to 180 TB.  VX-OS 3.5 lets existing customers scale capacity up to 64 NVMe SSDs, without re-cabling. Each array can support 16 storage blades that connect directly into the Vexata Ethernet midplane.

The upgrade expands VX-100 arrays to 435 TB in 6U. The system does not use an SSD cache for acceleration or inline data reduction for capacity. All capacity in the system is presented as raw storage.

VX-100 arrays include 16 ports for 32 Gbps Fibre Channel, NVMe over Fibre Channel or NVMe over lossless Ethernet. Each storage blade consists of four SSDs and embedded processing for local scheduling and metadata operations. Vexata supports clones, erasure coding, data protection with RAID 5 and RAID 6 and thin provisioning.

Hussain said customers will be able to get 1 PB in the 6U form factor by the end of 2018 by swapping in 8 TB

SSDs
.

“Conventional arrays still use x86 controllers and I/O paths that slow down the performance,” said Tim Stammers, a senior storage analyst at 451 Research in New York City. “The key thing Vexata is doing is to streamline the SAN architecture to get the most performance possible out of the NVMe storage.”

Vexata chief marketing officer, Ashish Gupta, said customers gain about 4 GBs of throughput when a blade is added to the array.

“We’re giving you the ability to scale performance and capacity with a single blade addition,” Gupta said. “That means you can maintain your cost structure without increasing a whole bunch of controllers, stitching them together, with the operational and computational complexities that come with that.”

The VX-OS release adds a feature for managing multiple Vexata chassis with a single pane.

About two dozen customers are using Vexata’s NVMe flash storage in production, Hussain said. They include the University of Hawai’i’s Pacific Disaster Center, payroll processing outfit Tata Consultancy Services, and Oath Inc., the digital media subsidiary of Verizon Communications.

“They have managed to get some production deployments already,” Stammers said. “There are quite a few types of application for very fast purpose-designed NVMe storage. The market isn’t moving at a rapid pace at the moment, but there is demand [growing] for this type of extremely fast, purpose-designed NVMe storage.”

How can technology empower the Class of 2030? |

Our world is changing faster than ever. What skills will today’s kindergarteners need to be life-ready by the time they graduate as the class of 2030? How can technology support their educational journey?

To answer these critical questions, we launched a key piece of research: “The class of 2030 and life-ready learning: The technology imperative.” Today, I am super excited to share that the full report is now available.

To conduct our research, we listened to 70 thought leaders around the world, reviewed 150 pieces of existing research, and surveyed 2,000 teachers and 2,000 students across Canada, Singapore, the United Kingdom and the United States.

Most importantly, we focused on the needs and aspirations of our subjects: the young people who make up the class of 2030 and those closest to understanding their world.

Within that context, we found 2 core themes: social emotional skills and personalized learning. Whilst not new in education, these are newly important for more people. Employers are placing a premium on social skills and emotional literacy with up to 40% of future jobs requiring explicit social emotional skills. Academics are noting their impact on deep learning and the students themselves recognize these skills are critical for success. The research highlighted personalized learning as an approach which supports skill development — both cognitive and social and emotional by guiding students towards greater autonomy and control.

The students were clear: they want to develop these skills to navigate their own learning – to explore and make choices that unlock their curiosity and potential and they want teachers who know and understand them as individuals.

Three technologies were highlighted in the research as showing great promise to support social and emotional skill development and personalized learning approaches; collaborative platforms, mixed reality and analytics powered by AI.

Students across the four surveyed countries prioritized a range of social-emotional and higher-order skills; notably, students valued digital skills, creativity and problem solving higher than teachers.

The students were clear: they want to develop the skills to navigate their own learning – to explore and make choices that unlock their curiosity and potential. Click To Tweet

While not new in education, these skills are newly important to more people and are taking center stage alongside deeper cognitive skills and content knowledge in the classroom and in the workforce. By 2030, it is predicted that between 30 to 40 percent of jobs will require explicit social-emotional skills.

While the need for social-emotional skills is clear, our research highlighted differences between the specific skills that students and teachers prioritize and how well-equipped teachers feel to teach these skills. This variation was mirrored in how both groups described their experiences of social-emotional skills as part of the learning program.

Students already place a strong emphasis on social-emotional skills based on the results of our survey, which you can explore below.

Personalized learning, which is a student centered approach, emerged as one of the most promising ways to develop social-emotional and deeper cognitive skills.

The students we surveyed said they want to have greater control over their learning and not just automation of content delivery. The students wanted personalization involving control over pacing, content and assessment. 70 percent of the students felt their mastery of content would be better with greater control. Quality feedback is critical for personalized learning. Only 40 percent of students we surveyed felt they receive feedback that was personalized; yet 60 percent of teachers felt they were providing personalized feedback.

Teachers have long endorsed personalized learning but have cited a lack of time and resources as obstacles. In our survey, nearly 70 percent of teachers cited time constraints as their biggest hurdle to providing more personalized content to their students. Our research revealed technology can help clear away those obstacles by freeing up as much as 30 percent of teachers’ time, so they can spend more time responding to individual and group needs.

Our research identified other important differences in perspectives between students and teachers about the extent of personalization in their current education.  Explore the data, below.

Three types of technology show especially strong promise for advancing socially embedded and personalized learning, and more immersive learning experiences: collaboration platforms, artificial intelligence, and mixed reality. You can find out more about the current and emerging opportunities in the full report and in the papers and case studies being published across 2018.

The class of 2030 and future generations will face social and global problems beyond what we can imagine. They will learn and engage with each other, with technology and with information in entirely new ways. And they will enter a workforce where job functions and roles will be dramatically different from today.

We hope our research will advance all of our efforts to help the class of 2030 be ready to succeed in work and life.

I encourage you to request our full report and share it widely with anyone interested in shaping the education system of the future.

I look forward to continuing to engage with you about these crucial topics in the weeks and months ahead.

95 Best Practices for Optimizing Hyper-V Performance

We can never get enough performance. Everything needs to be faster, faster, faster! You can find any number of articles about improving Hyper-V performance and best practices, of course, unfortunately, a lot of the information contains errors, FUD, and misconceptions. Some are just plain dated. Technology has changed and experience is continually teaching us new insights. From that, we can build a list of best practices that will help you to tune your system to provide maximum performance.

How to boost hyper-V performance - 95 best practices

Philosophies Used in this Article

This article focuses primarily on performance. It may deviate from other advice that I’ve given in other contexts. A system designed with performance in mind will be built differently from a system with different goals. For instance, a system that tries to provide high capacity at a low price point would have a slower performance profile than some alternatives.

  • Subject matter scoped to the 2012 R2 and 2016 product versions.
  • I want to stay on target by listing the best practices with fairly minimal exposition. I’ll expand ideas where I feel the need; you can always ask questions in the comments section.
  • I am not trying to duplicate pure physical performance in a virtualized environment. That’s a wasted effort.
  • I have already written an article on best practices for balanced systems. It’s a bit older, but I don’t see anything in it that requires immediate attention. It was written for the administrator who wants reasonable performance but also wants to stay under budget.
  • This content targets datacenter builds. Client Hyper-V will follow the same general concepts with variable applicability.

General Host Architecture

If you’re lucky enough to be starting in the research phase — meaning, you don’t already have an environment — then you have the most opportunity to build things properly. Making good purchase decisions pays more dividends than patching up something that you’ve already got.

  1. Do not go in blind.
    • Microsoft Assessment and Planning Toolkit will help you size your environment: MAP Toolkit
    • Ask your software vendors for their guidelines for virtualization on Hyper-V.
    • Ask people that use the same product(s) if they have virtualized on Hyper-V.
  2. Stick with logo-compliant hardware. Check the official list: https://www.windowsservercatalog.com/
  3. Most people will run out of memory first, disk second, CPU third, and network last. Purchase accordingly.
  4. Prefer newer CPUs, but think hard before going with bleeding edge. You may need to improve performance by scaling out. Live Migration requires physical CPUs to be the same or you’ll need to enable CPU compatibility mode. If your environment starts with recent CPUs, then you’ll have the longest amount of time to be able to extend it. However, CPUs commonly undergo at least one revision, and that might be enough to require compatibility mode. Attaining maximum performance may reduce virtual machine mobility.
  5. Set a target density level, e.g. “25 virtual machines per host”. While it may be obvious that higher densities result in lower performance, finding the cut-off line for “acceptable” will be difficult. However, having a target VM number in mind before you start can make the challenge less nebulous.
  6. Read the rest of this article before you do anything.

Management Operating System

Before we carry on, I just wanted to make sure to mention that Hyper-V is a type 1 hypervisor, meaning that it runs right on the hardware. You can’t “touch” Hyper-V because it has no direct interface. Instead, you install a management operating system and use that to work with Hyper-V. You have three choices:

Note: Nano Server initially offered Hyper-V, but that functionality will be removed (or has already been removed, depending on when you read this). Most people ignore the fine print of using Nano Server, so I never recommended it anyway.

TL;DR: In absence of a blocking condition, choose Hyper-V Server. A solid blocking condition would be the Automatic Virtual Machine Activation feature of Datacenter Edition. In such cases, the next preferable choice is Windows Server in Core mode.

I organized those in order by distribution size. Volumes have been written about the “attack surface” and patching. Most of that material makes me roll my eyes. No matter what you think of all that, none of it has any meaningful impact on performance. For performance, concern yourself with the differences in CPU and memory footprint. The widest CPU/memory gap lies between Windows Server and Windows Server Core. When logged off, the Windows Server GUI does not consume many resources, but it does consume some. The space between Windows Server Core and Hyper-V Server is much tighter, especially when the same features/roles are enabled.

One difference between Core and Hyper-V Server is the licensing mechanism. On Datacenter Edition, that does include the benefit of Automatic Virtual Machine Activation (AVMA). That only applies to the technological wiring. Do not confuse it with the oft-repeated myth that installing Windows Server grants guest licensing privileges. The legal portion of licensing stands apart; read our eBook for starting information.

Because you do not need to pay for the license for Hyper-V Server, it grants one capability that Windows Server does not: you can upgrade at any time. That allows you to completely decouple the life cycle of your hosts from your guests. Such detachment is a hallmark of the modern cloud era.

If you will be running only open source operating systems, Hyper-V Server is the natural choice. You don’t need to pay any licensing fees to Microsoft at all with that usage. I don’t realistically expect any pure Linux shops to introduce a Microsoft environment, but Linux-on-Hyper-V is a fantastic solution in a mixed-platform environment. And with that, let’s get back onto the list.

Management Operating System Best Practices for Performance

  1. Prefer Hyper-V Server first, Windows Server Core second
  2. Do not install any software, feature, or role in the management operating system that does not directly aid the virtual machines or the management operating system. Hyper-V prioritizes applications in the management operating system over virtual machines. That’s because it trusts you; if you are running something in the management OS, it assumes that you really need it.
  3. Do not log on to the management operating system. Install the management tools on your workstation and manipulate Hyper-V remotely.
  4. If you must log on to the management operating system, log off as soon as you’re done.
  5. Do not browse the Internet from the management operating system. Don’t browse from any server, really.
  6. Stay current on mainstream patches.
  7. Stay reasonably current on driver versions. I know that many of my peers like to install drivers almost immediately upon release, but I can’t join that camp. While it’s not entirely unheard of for a driver update to bring performance improvements, it’s not common. With all of the acquisitions and corporate consolidations going on in the hardware space — especially networking — I feel that the competitive drive to produce quality hardware and drivers has entered a period of decline. In simple terms, view new drivers as a potential risk to stability, performance, and security.
  8. Join your hosts to the domain. Systems consume less of your time if they answer to a central authority.
  9. Use antivirus and intrusion prevention. As long you choose your anti-malware vendor well and the proper exclusions are in place, performance will not be negatively impacted. Compare that to the performance of a compromised system.
  10. Read through our article on host performance tuning.

Leverage Containers

In the “traditional” virtualization model, we stand up multiple virtual machines running individual operating system environments. As “virtual machine sprawl” sets in, we wind up with a great deal of duplication. In the past, we could justify that as a separation of the environment. Furthermore, some Windows Server patches caused problems for some software but not others. In the modern era, containers and omnibus patch packages have upset that equation.

Instead of building virtual machine after virtual machine, you can build a few virtual machines. Deploy containers within them. Strategies for this approach exceed the parameters of this article, but you’re aiming to reduce the number of disparate complete operating system environments deployed. With careful planning, you can reduce density while maintaining a high degree of separation for your services. Fewer kernels are loaded, fewer context switches occur, less memory contains the same code bits, fewer disk seeks to retrieve essentially the same information from different locations.

  1. Prefer containers over virtual machines where possible.

CPU

You can’t do a great deal to tune CPU performance in Hyper-V. Overall, I count that among my list of “good things”; Microsoft did the hard work for you.

  1. Follow our article on host tuning; pay special attention to C States and the performance power settings.
  2. For Intel chips, leave hyperthreading on unless you have a defined reason to turn it off.
  3. Leave NUMA enabled in hardware. On your VMs’ property sheet, you’ll find a Use Hardware Topology button. Remember to use that any time that you adjust the number of vCPUs assigned to a virtual machine or move it to a host that has a different memory layout (physical core count and/or different memory distribution).
    best pratices for optimizing hyper-v performance - settings NUMA configuration
  4. Decide whether or not to allow guests to span NUMA nodes (the global host NUMA Spanning setting). If you size your VMs to stay within a NUMA node and you are careful to not assign more guests than can fit solidly within each NUMA node, then you can increase individual VM performance. However, if the host has trouble locking VMs into nodes, then you can negatively impact overall memory performance. If you’re not sure, just leave NUMA at defaults and tinker later.
  5. For modern guests, I recommend that you use at least two virtual CPUs per virtual machine. Use more in accordance with the virtual machine’s performance profile or vendor specifications. This is my own personal recommendation; I can visibly detect the response difference between a single vCPU guest and a dual vCPU guest.
  6. For legacy Windows guests (Windows XP/Windows Server 2003 and earlier), use 1 vCPU. More will likely hurt performance more than help.
  7. Do not grant more than 2 vCPU to a virtual machine without just cause. Hyper-V will do a better job reducing context switches and managing memory access if it doesn’t need to try to do too much core juggling. I’d make exceptions for very low-density hosts where 2 vCPU per guest might leave unused cores. At the other side, if you’re assigning 24 cores to every VM just because you can, then you will hurt performance.
  8. If you are preventing VMs from spanning NUMA nodes, do not assign more vCPU to a VM than you have matching physical cores in a NUMA node (usually means the number of cores per physical socket, but check with your hardware manufacturer).
  9. Use Hyper-V’s priority, weight, and reservation settings with great care. CPU bottlenecks are highly uncommon; look elsewhere first. A poor reservation will cause more problems than it solves.

Memory

I’ve long believed that every person that wants to be a systems administrator should be forced to become conversant in x86 assembly language, or at least C. I can usually spot people that have no familiarity with programming in such low-level languages because they almost invariably carry a bizarre mental picture of how computer memory works. Fortunately, modern memory is very, very, very fast. Even better, the programmers of modern operating system memory managers have gotten very good at their craft. Trying to tune memory as a systems administrator rarely pays dividends. However, we can establish some best practices for memory in Hyper-V.

  1. Follow our article on host tuning. Most importantly, if you have multiple CPUs, install your memory such that it uses multi-channel and provides an even amount of memory to each NUMA node.
  2. Be mindful of operating system driver quality. Windows drivers differ from applications in that they can permanently remove memory from the available pool. If they do not properly manage that memory, then you’re headed for some serious problems.
  3. Do not make your CSV cache too large.
  4. For virtual machines that will perform high quantities of memory operations, avoid dynamic memory. Dynamic memory disables NUMA (out of necessity). How do you know what constitutes a “high volume”? Without performance monitoring, you don’t.
  5. Set your fixed memory VMs to a higher priority and a shorter startup delay than your Dynamic Memory VMs. This ensures that they will start first, allowing Hyper-V to plot an optimal NUMA layout and reduce memory fragmentation. It doesn’t help a lot in a cluster, unfortunately. However, even in the best case, this technique won’t yield many benefits.
  6. Do not use more memory for a virtual machine than you can prove that it needs. Especially try to avoid using more memory than will fit in a single NUMA node.
  7. Use Dynamic Memory for virtual machines that do not require the absolute fastest memory performance.
  8. For Dynamic Memory virtual machines, pay the most attention to the startup value. It sets the tone for how the virtual machine will be treated during runtime. For virtual machines running full GUI Windows Server, I tend to use a startup of either 1 GB or 2 GB, depending on the version and what else is installed.
  9. For Dynamic Memory VMs, set the minimum to the operating system vendor’s stated minimum (512 MB for Windows Server). If the VM hosts a critical application, add to the minimum to ensure that it doesn’t get choked out.
  10. For Dynamic Memory VMs, set the maximum to a reasonable amount. You’ll generally discover that amount through trial and error and performance monitoring. Do not set it to an arbitrarily high number. Remember that, even on 2012 R2, you can raise the maximum at any time.

Check the CPU section for NUMA guidance.

Networking

In the time that I’ve been helping people with Hyper-V, I don’t believe that I’ve seen anyone waste more time worrying about anything that’s less of an issue than networking. People will read whitepapers and forums and blog articles and novels and work all weekend to draw up intricately designed networking layouts that need eight pages of documentation. But, they won’t spend fifteen minutes setting up a network utilization monitor. I occasionally catch grief for using MRTG since it’s old and there are shinier, bigger, bolder tools, but MRTG is easy and quick to set up. You should know how much traffic your network pushes. That knowledge can guide you better than any abstract knowledge or feature list.

That said, we do have many best practices for networking performance in Hyper-V.

  1. Follow our article on host tuning. Especially pay attention to VMQ on gigabit and separation of storage traffic.
  2. If you need your network to go faster, use faster adapters and switches. A big team of gigabit won’t keep up with a single 10 gigabit port.
  3. Use a single virtual switch per host. Multiple virtual switches add processing overhead. Usually, you can get a single switch to do whatever you wanted multiple switches to do.
  4. Prefer a single large team over multiple small teams. This practice can also help you to avoid needless virtual switches.
  5. For gigabit, anything over 4 physical ports probably won’t yield meaningful returns. I would use 6 at the outside. If you’re using iSCSI or SMB, then two more physical adapters just for that would be acceptable.
  6. For 10GbE, anything over 2 physical ports probably won’t yield meaningful returns.
  7. If you have 2 10GbE and a bunch of gigabit ports in the same host, just ignore the gigabit. Maybe use it for iSCSI or SMB, if it’s adequate for your storage platform.
  8. Make certain that you understand how the Hyper-V virtual switch functions. Most important:
    • You cannot “see” the virtual switch in the management OS except with Hyper-V specific tools. It has no IP address and no presence in the Network and Sharing Center applet.
    • Anything that appears in Network and Sharing Center that you think belongs to the virtual switch is actually a virtual network adapter.
    • Layer 3 (IP) information in the host has no bearing on guests — unless you create an IP collision
  9. Do not create a virtual network adapter in the management operating system for the virtual machines. I did that before I understood the Hyper-V virtual switch, and I have encountered lots of other people that have done it. The virtual machines will use the virtual switch directly.
  10. Do not multi-home the host unless you know exactly what you are doing. Valid reasons to multi-home:
    • iSCSI/SMB adapters
    • Separate adapters for cluster roles. e.g. “Management”, “Live Migration”, and “Cluster Communications”
  11. If you multi-home the host, give only one adapter a default gateway. If other adapters must use gateways, use the old route command or the new New-NetRoute command.
  12. Do not try to use internal or private virtual switches for performance. The external virtual switch is equally fast. Internal and private switches are for isolation only.
  13. If all of your hardware supports it, enable jumbo frames. Ensure that you perform validation testing (i.e.:
    ping storage-ip -f -l 8000)
  14. Pay attention to IP addressing. If traffic needs to locate an external router to reach another virtual adapter on the same host, then traffic will traverse the physical network.
  15. Use networking QoS if you have identified a problem.
    • Use datacenter bridging, if your hardware supports it.
    • Prefer the Weight QoS mode for the Hyper-V switch, especially when teaming.
    • To minimize the negative side effects of QoS, rely on limiting the maximums of misbehaving or non-critical VMs over trying to guarantee minimums for vital VMs.
  16. If you have SR-IOV-capable physical NICs, it provides the best performance. However, you can’t use the traditional Windows team for the physical NICs. Also, you can’t use VMQ and SR-IOV at the same time.
  17. Switch-embedded teaming (2016) allows you to use SR-IOV. Standard teaming does not.
  18. If using VMQ, configure the processor sets correctly.
  19. When teaming, prefer Switch Independent mode with the Dynamic load balancing algorithm. I have done some performance testing on the types (near the end of the linked article). However, a reader commented on another article that the Dynamic/Switch Independent combination can cause some problems for third-party load balancers (see comments section).

Storage

When you need to make real differences in Hyper-V’s performance, focus on storage. Storage is slow. The best way to make storage not be slow is to spend money. But, we have other ways.

  1. Follow our article on host tuning. Especially pay attention to:
    • Do not break up internal drive bays between Hyper-V and the guests. Use one big array.
    • Do not tune the Hyper-V partition for speed. After it boots, Hyper-V averages zero IOPS for itself. As a prime example, don’t put Hyper-V on SSD and the VMs on spinning disks. Do the opposite.
    • The best ways to get more storage speed is to use faster disks and bigger arrays. Almost everything else will only yield tiny differences.
  2. For VHD (not VHDX), use fixed disks for maximum performance. Dynamically-expanding VHD is marginally, but measurably, slower.
  3. For VHDX, use dynamically-expanding disks for everything except high-utilization databases. I receive many arguments on this, but I’ve done the performance tests and have years of real-world experience. You can trust that (and run the tests yourself), or you can trust theoretical whitepapers from people that make their living by overselling disk space but have perpetually misplaced their copy of diskspd.
  4. Avoid using shared VHDX (2012 R2) or VHDS (2016). Performance still isn’t there. Give this technology another maturation cycle or two and look at it again.
  5. Where possible, do not use multiple data partitions in a single VHD/X.
  6. When using Cluster Shared Volumes, try to use at least as many CSVs as you have nodes. Starting with 2012 R2, CSV ownership will be distributed evenly, theoretically improving overall access.
  7. You can theoretically improve storage performance by dividing virtual machines across separate storage locations. If you need to make your arrays span fewer disks in order to divide your VMs’ storage, you will have a net loss in performance. If you are creating multiple LUNs or partitions across the same disks to divide up VMs, you will have a net loss in performance.
  8. For RDS virtual machine-based VDI, use hardware-based or Windows’ Hyper-V-mode deduplication on the storage system. The read hits, especially with caching, yield positive performance benefits.
  9. The jury is still out on using host-level deduplication for Windows Server guests, but it is supported with 2016. I personally will be trying to place Server OS disks on SMB storage deduplicated in Hyper-V mode.
  10. The slowest component in a storage system is the disk(s); don’t spend a lot of time worrying about controllers beyond enabling caching.
  11. RAID-0 is the fastest RAID type, but provides no redundancy.
  12. RAID-10 is generally the fastest RAID type that provides redundancy.
  13. For Storage Spaces, three-way mirror is fastest (by a lot).
  14. For remote storage, prefer MPIO or SMB multichannel over multiple unteamed adapters. Avoid placing this traffic on teamed adapters.
  15. I’ve read some scattered notes that say that you should format with 64 kilobyte allocation units. I have never done this, mostly because I don’t think about it until it’s too late. If the default size hurts anything, I can’t tell. Someday, I’ll remember to try it and will update this article after I’ve gotten some performance traces. If you’ll be hosting a lot of SQL VMs and will be formatting their VHDX with 64kb AUs, then you might get more benefit.
  16. I still don’t think that ReFS is quite mature enough to replace NTFS for Hyper-V. For performance, I definitely stick with NTFS.
  17. Don’t do full defragmentation. It doesn’t help. The minimal defragmentation that Windows automatically performs is all that you need. If you have some crummy application that makes this statement false, then stop using that application or exile it to its own physical server. Defragmentation’s primary purpose is to wear down your hard drives so that you have to buy more hard drives sooner than necessary, which is why employees of hardware vendors recommend it all the time. If you have a personal neurosis that causes you pain when a disk becomes “too” fragmented, use Storage Live Migration to clear and then re-populate partitions/LUNs. It’s wasted time that you’ll never get back, but at least it’s faster. Note: All retorts must include verifiable and reproducible performance traces, or I’m just going to delete them.

Clustering

For real performance, don’t cluster virtual machines. Use fast internal or direct-attached SSDs. Cluster for redundancy, not performance. Use application-level redundancy techniques instead of relying on Hyper-V clustering.

In the modern cloud era, though, most software doesn’t have its own redundancy and host clustering is nearly a requirement. Follow these best practices:

  1. Validate your cluster. You may not need to fix every single warning, but be aware of them.
  2. Follow our article on host tuning. Especially pay attention to the bits on caching storage. It includes a link to enable CSV caching.
  3. Remember your initial density target. Add as many nodes as necessary to maintain that along with sufficient extra nodes for failure protection.
  4. Use the same hardware in each node. You can mix hardware, but CPU compatibility mode and mismatched NUMA nodes will have at least some impact on performance.
  5. For Hyper-V, every cluster node should use a minimum of two separate IP endpoints. Each IP must exist in a separate subnet. This practice allows the cluster to establish multiple simultaneous network streams for internode traffic.
    • One of the addresses must be designated as a “management” IP, meaning that it must have a valid default gateway and register in DNS. Inbound connections (such as your own RDP and PowerShell Remoting) will use that IP.
    • None of the non-management IPs should have a default gateway or register in DNS.
    • One alternative IP endpoint should be preferred for Live Migration. Cascade Live Migration preference order through the others, ending with the management IP. You can configure this setting most easily in Failover Cluster Manager by right-clicking on the Networks node.
    • Further IP endpoints can be used to provide additional pathways for cluster communications. Cluster communications include the heartbeat, cluster status and update messages, and Cluster Shared Volume information and Redirected Access traffic.
    • You can set any adapter to be excluded from cluster communications but included in Live Migration in order to enforce segregation. Doing so generally does not improve performance, but may be desirable in some cases.
    • You can use physical or virtual network adapters to host cluster IPs.
    • The IP for each cluster adapter must exist in a unique subnet on that host.
    • Each cluster node must contain an IP address in the same subnet as the IPs on other nodes. If a node does not contain an IP in a subnet that exists on other nodes, then that network will be considered “partitioned” and the node(s) without a member IP will be excluded from that network.
    • If the host will connect to storage via iSCSI, segregate iSCSI traffic onto its own IP(s). Exclude it/them from cluster communications and Live Migration. Because they don’t participate in cluster communications, it is not absolutely necessary that they be placed into separate subnets. However, doing so will provide some protection from network storms.
  6. If you do not have RDMA-capable physical adapters, Compression usually provides the best Live Migration performance.
  7. If you do have RDMA-capable physical adapters, SMB usually provides the best Live Migration performance.
  8. I don’t recommend spending time tinkering with the metric to shape CSV traffic anymore. It utilizes SMB, so the built-in SMB multi-channel technology can sort things out.

Virtual Machines

The preceding guidance obliquely covers several virtual machine configuration points (check the CPU and the memory sections). We have a few more:

  1. Don’t use Shielded VMs or BitLocker. The encryption and VMWP hardening incur overhead that will hurt performance. The hit is minimal — but this article is about performance.
  2. If you have 1) VMs with very high inbound networking needs, 2) physical NICs >= 10GbE, 3) VMQ enabled, 4) spare CPU cycles, then enable RSS within the guest operating systems. Do not enable RSS in the guest OS unless all of the preceding are true.
  3. Do not use the legacy network adapter in Generation 1 VMs any more than absolutely necessary.
  4. Utilize checkpoints rarely and briefly. Know the difference between standard and “production” checkpoints.
  5. Use time synchronization appropriately. Meaning, virtual domain controllers should not have the Hyper-V time synchronization service enabled, but all other VMs should (generally speaking). The hosts should pull their time from the domain hierarchy. If possible, the primary domain controller should be pulling from a secured time source.
  6. Keep Hyper-V guest services up-to-date. Supported Linux systems can be updated via kernel upgrades/updates from their distribution repositories. Windows 8.1+ and Windows Server 2012 R2+ will update from Windows Update.
  7. Don’t do full defragmentation in the guests, either. Seriously. We’re administering multi-spindle server equipment here, not displaying a progress bar to someone with a 5400-RPM laptop drive so that they feel like they’re accomplishing something.
  8. If the virtual machine’s primary purpose is to run an application that has its own replication technology, don’t use Hyper-V Replica. Examples: Active Directory and Microsoft SQL Server. Such applications will replicate themselves far more efficiently than Hyper-V Replica.
  9. If you’re using Hyper-V Replica, consider moving the VMs’ page files to their own virtual disk and excluding it from the replica job. If you have a small page file that doesn’t churn much, that might cost you more time and effort than you’ll recoup.
  10. If you’re using Hyper-V Replica, enable compression if you have spare CPU but leave it disabled if you have spare network. If you’re not sure, use compression.
  11. If you are shipping your Hyper-V Replica traffic across an encrypted VPN or keeping its traffic within secure networks, use Kerberos. SSL en/decryption requires CPU. Also, the asymmetrical nature of SSL encryption causes the encrypted data to be much larger than its decrypted source.

Monitoring

You must monitor your systems. Monitoring is not and has never been, an optional activity.

  1. Be aware of Hyper-V-specific counters. Many people try to use Task Manager in the management operating system to gauge guest CPU usage, but it just doesn’t work. The management operating system is a special-case virtual machine, which means that it is using virtual CPUs. Its Task Manager cannot see what the guests are doing.
  2. Performance Monitor has the most power of any built-in tool, but it’s tough to use. Look at something like Performance Analysis of Logs (PAL) tool, which understands Hyper-V.
  3. In addition to performance monitoring, employ state monitoring. With that, you no longer have to worry (as much) about surprise events like disk space or memory filling up. I like Nagios, as regular readers already know, but you can select from many packages.
  4. Take periodic performance baselines and compare them to earlier baselines

If you’re able to address a fair proportion of points from this list, I’m sure you’ll see a boost in Hyper-V performance. Don’t forget this list is not exhaustive and I’ll be adding to it periodically to ensure it’s as comprehensive as possible however if you think there’s something missing, let me know in the comments below and you may see the number 95 increase!

Java 18.3 marks the future of Java at JavaOne 2017

SAN FRANCISCO — While Oracle has committed to a faster cadence for releasing Java technology every six months, the company also is making sure to maintain a pipeline of new technology that’s ready to go in each release.

The next major release of Java, known as Java 18.3, is scheduled for March 2018 and will feature some of the innovations coming out of projects Oracle is currently working on that will affect the future of Java.

During his keynote at the JavaOne 2017 conference, Mark Reinhold, chief architect of the Java platform group at Oracle, identified four projects the company is working on that will bear fruit for Java 18.3 and subsequent releases.

Four projects will affect the future of Java

Reinhold identified Project Valhalla, Project Panama, Project Amber and Project Loom as key initiatives from which Java developers can expect to gain innovation in the not-too-distant future.

According to Reinhold, Project Panama is about foreign function interface data-layout control. It seeks to improve the connection between Java and native data and native code. The project focuses on interconnecting the Java Virtual Machine (JVM) and native code — a key concept for the future of Java. A description of the project on Oracle’s website said it is about enriching the connections between the JVM and well-defined but “foreign” or non-Java APIs, including many interfaces commonly used by C programmers.

The goal of Project Valhalla is to explore advanced Java Virtual Machine and language features, such as value types and generic specialization.

Meanwhile, Project Amber is about right-sizing language ceremony, he said. It’s about delivering smaller, productivity-oriented Java language features, such as local-variable type inference, enhanced enums, Lambda leftovers and more.

And Project Loom is aimed at bringing continuations and new features known as fibers to the Java platform to simplify concurrency for developers. Reinhold said Project Loom has not yet been proposed, but is currently in discussion as an effort to affect the future of Java for concurrent programming.

Solid case made for mature integration

According to Charlotte Dunlap, an analyst with GlobalData, “Many of Oracle’s announcements were actually preannouncements. However, Oracle made a solid case for how its mature integration, SOA and API management technologies will enable the next wave of Oracle’s DevOps technologies.”

Regarding Project Amber, Brian Goetz, a Java language architect in the Java platform group at Oracle, said Java has a reputation for being “a bit boilerplate-intensive,” and it takes a little too much code to do a lot of common tasks.

“So, Project Amber is a collection of smaller features that are aimed at reducing the overhead or ceremony of things that we do every day — streamlining everyday coding, but also making code more readable and more reliable,” he said.

These are features that can be delivered over time and are a good fit for the programming models that are popular in the cloud, such as functions as a service, or reactive event-based systems, like message-based systems or actors, Goetz said.

Amber adds variable type inference

Project Amber is a collection of smaller features that are aimed at reducing the overhead or ceremony of things that we do every day — streamlining everyday coding, but also making code more readable and more reliable.
Brian GoetzJava language architect at Oracle

Meanwhile, a subproject of Amber is called local variable type inference, which Goetz said is a feature that has been available in a number of other programming languages. And although Java has had type inference for many years, it’s being expanded.

“Type inference is basically just the compiler figuring out the type of something without you having to write it down,” Goetz said. “It can make code more readable by getting unnecessary information out of the way. And this next iteration is extending that to the way we declare local variables.”

This feature has been committed to Java Development Kit 18.3 and will be in the next production release of Java in March.

However, a much bigger feature in Project Amber, known as pattern matching, has been historically associated with functional programming languages, Goetz said.

More recently, it has been adopted by object-oriented programming languages, like C# and Scala. “We think it’s a really good fit for Java,” Goetz noted. “It has the ability to simplify the kind of code where you have to do multiway conditional operations.”

Project Loom to ease app maintenance

Finally, as described in Oracle software engineer Ron Pressler’s call for discussion about the initiative, Project Loom is an effort to provide an alternative implementation of threads, managed by schedulers written in Java, that preserve the same programming model of ordinary Java threads, but offer drastically improved performance and a smaller footprint.

According to the Project Loom proposal, “Project Loom’s mission is to make it easier to write, debug, profile and maintain concurrent applications meeting today’s requirements. Threads, provided by Java from its first day, are a natural and convenient concurrency construct (putting aside the separate question of communication among threads) which is being supplanted by less convenient abstractions because their current implementation as OS kernel threads is insufficient for meeting modern demands, and wasteful in computing resources that are particularly valuable in the cloud. Project Loom will introduce fibers as lightweight, efficient threads managed by the Java Virtual Machine, that let developers use the same simple abstraction but with better performance and lower footprint.”

3 Emerging Technologies that Will Change the Way You Use Hyper-V

Hello once again everyone!

The I.T. landscape changes incredibly quickly (if you know a faster changing industry, I’d love to know!) I.T. professionals need to know what’s coming round the corner to stay ahead of the game or risk being left behind. Well, we don’t want that to happen to you, so we’ve run down what we feel are the three most important emerging technologies that will drastically change the Hyper-V landscape.

  1. Continued Adoption of Public Cloud Platforms – It’s becoming clear that the public cloud is continuing to gain steam. It’s not just one vendor, but several, and it continues to pull workloads from on-premise to the cloud. Many people were keen to wait out this “cloud thing”, but it has become quite clear that it’s here to stay. Capabilities in online platforms such as Microsoft Azure and Amazon AWS, have increasingly made it easier, more cost-effective, and desirable to put workloads in the public cloud. These cloud platforms can often provide services that most customers don’t have available on-premise, and this paired with several other things that we’ll talk about in the webinar are leading to increased adoption of these platforms over on-premise installations.
  2. Azure Stack and the Complete Abstraction of Hyper-V under-the-hood – With some of the latest news and release information out of Microsoft regarding their new Microsoft Azure Stack (MAS), things have taken an interesting turn for Hyper-V. As on-premise administrators have always been used to having direct access to the hypervisor, they may be surprised to learn that Hyper-V is so far under the hood in MAS that you can’t even access it. That’s right. The Hypervisor has become so simplified and automated, that there is no need to directly access it in MAS, but this is primarily because MAS follows the same usage and management guidelines as Microsoft Azure. This will bother a lot of administrators but it’s becoming the world we live in. As such, we’ll be talking about this extensively during the webinar.
  3. Containers and Microservices and why they are a game-changer – Containers has become one of the new buzz-words in the industry. If you’re not aware, you can think of containers as similar to a VM, but fundamentally different. Whereas in a VM you’re virtualizing the OS, and everything on top of it, with containers you’re only virtualizing the application. Much of the underlying support functions are handled by the container host, as opposed to an OS built into a VM. For a long time it seemed that containers were going to primarily be a developer thing, but as the line between IT Pro and Dev continues to blur, Containers can no longer be ignored by IT Pros, and we’ll be talking about that revelation extensively during our panel discussion.

As you can see there is much to talk about, and many will be wondering how this affects them. You’re probably asking yourself questions like: “What new skills should IT Pros be learning to stay relevant?”, “Are hypervisors becoming irrelevant?”, “Will containers replace virtual machines?”, “Is the Cloud here to stay?”, “Is there still a place for Windows Server in the world?”, “What can I do now to stay relevant and what skills do I need to learn to future-proof my career?” Yep, these developments certainly raise a lot of issues which is why we decided to take this topic further.

Curious to know more? Join our Live Webinar!

As you know we love to put on webinars here at Altaro as we find them a critical tool for getting information about new technologies and features to our viewership. We’ve always stuck to the same basic educational format and it’s worked well over the years. However, we’ve always wanted to try something a bit different. There certainly isn’t anything wrong with an educational format, but with some topics, it’s often best to just have a conversation. This idea is at the core of our next webinar along with some critical changes that are occurring within our industry.

For the first time ever, Altaro will be putting on a panel-style webinar with not 1 or 2, but with 3 Microsoft Cloud and Datacenter MVPs. Andy Syrewicze, Didier Van Hoye, and Thomas Maurer will all be hosting this webinar as they talk about some major changes and take your questions/feedback regarding things that are occurring in the industry today. These are things that will affect the way you use and consume Hyper-V.

Webinar Registration

Artificial intelligence in software testing has arrived

What if software testing could undergo a metamorphosis, becoming better, faster and less expensive all the while letting testers focus on what they excel at? That rosy future could happen, thanks to a sudden interest in artificial intelligence in software testing.

Enterprise solutions provider Infostretch just announced it will offer artificial intelligence in software testing through a brand new service called Predictive and Prescriptive QA. Infostretch isn’t the only option — San Francisco-based startup Appdiff is also bringing machine learning “bots” online as testers. And dinCloud recently announced “James,” a virtual robot QA.

With continuous delivery, continuous integration and DevOps as the hot topics in every software development conversation today, the pressure on testers has never been more intense. “The thing is your crew cannot keep up with the amount of testing that should happen,” Appdiff CEO Jason Arbon said. “That’s one reason for Appdiff. … People can’t keep up any more.”

What about machine learning?

The solution is artificial intelligence in software testing, or more specifically, an AI subset: machine learning. “Today there are tons and tons of test data and it’s very hard for a single person to get through it all,” said Avery Lyford, chief customer officer at Infostretch. “It’s tons of report management now. Where are the real issues and what are the real problems?” That is where artificial intelligence in software testing can come in and help sort through the noise, Lyford said.

Infostretch is offering the Predictive and Prescriptive QA product as a service. With a heavy focus on data analysis, Lyford said the artificial intelligence in software testing tool can help streamline the testing process by ensuring the right information is in the hands of the testers who can then make better decisions. The new service can also be used in conjunction with the company’s QMetry offering.  

AppDiff is taking a slightly different approach, Arbon said. “We’re going from the end user experience backwards,” he said. “AI bots can do tens of thousands of test cases versus 20 to 100 regression test cases. This plays into today’s DevOps plan to iterate quickly.” Using artificial intelligence in software testing, companies will always know if the UI isn’t working or the UX is struggling, he said.

But these aren’t just any bots. Arbon, who previously worked at Google and has a background in software testing, realized a fundamental truth about applications that make bots effective testers. “Almost every app is the same,” he explained. “It’s the same log-in screen, most search boxes look the same, the profile, the shopping carts, there are a lot of similarities.” With that understanding — and the idea that each bot could be trained as a specialist in a single area like just the search box — Arbon was able to create bots that were better than the average tester. “The little bots are specialists on each area of the app and while they’re not as smart as a human might be, they’re the best search testers on the planet.” Arbon, and his colleagues who come from Google and Microsoft, train their bots to test like they did. “It’s like we’ve created a “Google tester” in a box. This replicates what we would do with your app.”

And, amazingly, there may be a silver lining in this for testers, many of whom fear being automated — or AI’d — out of a job. “The folks we work with don’t get fired,” Arbon said. “They get to hand off work and focus on doing things they’re good at.” Or to put it another way, it’s eliminating the grunt work and allowing testers to do the human, creative things they’re better at, said Paul Merrill, principal software engineer in test and founder of Beaufort Fairmont Automated Testing Services, at the Agile2017 conference in Orlando. Lyford sees it as giving testers back that elusive element of time. “We want people to be able to do complicated edge cases, not the routine stuff. This is to augment testers, not replace them.”

Artificial Intelligence and Formula One: Bots on pole position in the race for technology – Asia News Center

Not only would faster AI-integrated information boost race performance, it might also mean avoiding a crash or anticipating the failure of a crucial part inside a car’s power unit. And it would free up human crew members to do other tasks. d’Imbleval is planning for trackside AI to be in place next racing season.

“Artificial intelligence would be so important during a race,” he says. “We need help to take the best decisions that have to be taken during each lap time.”

ALSO READ: Artificial Intelligence in Asia: Here, there and everywhere

He sees AI as a logical next step in an ambitious digital transformation plan for Renault Sport Formula One Team, which is aiming to be on top of the F1 world three years from now.

‘’Next year, we will target podiums. And then in 2019 and 2020, we want to fight for the championship,” he said ahead of Singapore’s rain-affected night-time Grand Prix on September 17 when Renault Sport Formula One Team’s Jolyon Palmer finished in sixth place while teammate Nico Hulkenberg retired in the pits not long before the end.

d’Imbleval is unapologetically enthralled by the complexities and excitement of racing. He even describes himself as a “petrolhead” – a word he finds hard to translate into his native French.

Nonetheless, he comes from hard-headed corporate background. The team will only succeed, he says, if it runs as a  business that functions and delivers on time and on budget. To get there, Renault Sport Formula One Team has turned to longtime technology partner, Microsoft.

Next year, we will target podiums. And then in 2019 and 2020, we want to fight for the championship.

An array of Microsoft business solutions has already made its mark. Moving many operations into the cloud and using Microsoft Dynamics 365 solutions have better integrated the administration, coordination, and production of its two main design and manufacturing plants in Viry-Châtillon in France, and Enstone in England.

Renault Sport Formula One Team has four decades of experience in F1. It returned as a full Constructor in 2016 when it took over the Enstone outfit and began the journey of taking it back to the glory of 2005 and 2006 when the team, back then under Renault ownership, secured two consecutive Drivers and Constructors Championships.

Technology has also moved the needle in design and development. Azure Machine Learning helps predict the effects of changing car configurations. The hybrid capabilities of the Microsoft Cloud also come into play when Azure Stream Analytics syncs with Renault’s own supercomputer as it conducts 3D virtual testing of car designs to lessen the need for full-on wind tunnel tests.