Category Archives: mixed reality

Auto Added by WPeMatico

Mixed Reality @ Microsoft – July 2018 Update – Microsoft Devices Blog

Hey, everyone!
It’s been a busy summer here in Redmond, in particular the month of July – with our company fresh off Microsoft Inspire, our second Mixed Reality Partner Summit and the inaugural Microsoft Business Applications Summit. Every year July carries a special resonance at Microsoft. We’re closing one fiscal year and starting another, and it’s a great time spent with our global partners, field teams and customers.
What makes July meaningful are the engagements we have with our customers, partners, and developers. This passionate and dedicated community helps us fine tune what we have been engineering and working to bring to market. The insights we get are invaluable, and I always look forward to this time as it represents an important milestone in making our vision real for customers around the world.
With that, I want to take a moment and share some of what we have been working on, where we have been, and some really cool demos that highlight the ongoing opportunity we are seeing with Microsoft HoloLens and mixed reality business applications.
Microsoft Inspire
I had the opportunity to represent all the people on my team who are working to bring mixed reality business applications to life when I joined Satya Nadella’s Corenote address at Inspire. For those who don’t know about Inspire, it is Microsoft’s annual worldwide partner conference. This year more than 15,000 partners joined us in Las Vegas to talk about the year ahead. The full Corenote is an excellent overview on how Satya is thinking about our shared opportunity, as well as the technical and cultural leadership he’s setting for the company. But, if you just want to watch the mixed reality business applications part, I come on at the 56:55 mark.
I am passionate about the potential mixed reality presents to empower Firstline Workers across the globe. In fact, there are more than two billion Firstline Workers worldwide. That’s more than the number of Information Workers, and it presents a tremendous challenge to foster technology that gives them the tools they need to do their jobs in the field. Our current mixed reality business applications – Microsoft Remote Assist and Microsoft Layout – provide Firstline Workers the ability to work heads-up and hands-free with HoloLens, giving them a set of tools they have never had before to do real work and solve real problems.
When you combine that with the digitization of business processes via Dynamics 365, the digitization of productivity workflows with Microsoft 365, and of real-world data via Azure IoT (all connected by a Common Data Service), you are able to create experiences that will help serve as a catalyst for the future of work.
What you saw in the Inspire Corenote is a glimpse of this future. It’s just the beginning for how we plan to deeply integrate Dynamics 365 with our Mixed Reality Business Applications and with Microsoft 365 and Azure. We are clearly hearing from our customers and seeing firsthand that the combination of mixed reality and the existing strength of Microsoft’s Business Applications like Dynamics 365, PowerBI and the PowerApps platform will generate new ways for Firstline Workers to visualize information, consult data in real time and collaborate remotely, opening up an unprecedented amount of new opportunities.
Mixed Reality Partner Summit
In addition to Microsoft Inspire, our trip to Las Vegas brought with it our annual Mixed Reality Partner Program (MRPP) Summit. I love Inspire, and I REALLY LOVE the MRPP Summit! This is where we get to spend time with partners who work with customers to deploy HoloLens, custom applications, and mixed reality business applications. This is where we get to hear directly from the people who help us create and deliver the incredibly potential of mixed reality for customers.
At this point in our lifecycle, it’s amazing to see the growth and momentum for HoloLens. We started shipping just a little over two years ago. Hundreds of companies around the world have deployed HoloLens and as a result, thousands of workers around the world are using HoloLens in their daily workflows. What an awesome accomplishment and responsibility for all of us!
The MRPP has grown a ton since the early days (meaning last year ????). It now includes more than 100 companies, including ISVs (Independent Software Vendors), SIs (Systems Integrators), distributors (who resell HoloLens), creative agencies, and more.
In many ways the MRPP serves as the foundation of our community. They are fostering the creativity that is helping bring mixed reality to life and it’s inspiring to spend time with them, thinking about and planning for the future.
Microsoft Business Applications Summit
July also saw us welcome thousands of customers and partners to our inaugural Microsoft Business Applications Summit in Seattle. This community came together with our engineering teams that build Dynamics 365, Power BI, PowerApps, Microsoft Flow and mixed reality business applications to connect, share, learn and be the first to know where we’re going and what’s coming next across the business applications landscape.
This event is all about making the community more successful. The Business Applications Summit was anchored with a keynote from by James Phillips, corporate vice president, Business Applications Group. James talked about how Microsoft’s business applications are forging new opportunities and highlighted cutting-edge work from partners and discussed mixed reality. It was awesome to see how Microsoft and partners are working to bring forward solutions that integrate Microsoft’s three clouds – Dynamics 365, Microsoft 365, and Azure – with mixed reality and IoT, all thanks to the Common Data Service.
On stage we saw an example that brings together field service and finance and operations with mixed reality and IoT capabilities. This type of work will empower a Firstline Worker with real-time collaboration and real-time access to IoT data throughout the workflow.
We showed how a company like MacDonald-Miller can take advantage of this connected workflow. We started with field service, which is actively listening all the time to those IoT signals and then proactively dispatches a technician to solve the problem. Then we showed how, with HoloLens and Microsoft Remote Assist, that Firstline Worker can call a remote expert anytime from any place, and then we ended with how the information flows right into finance and operations.
Check out our demo with MacDonald-Miller (around the 50:00 mark).
I’m looking forward to next month and what’s ahead as we get ready for our next customer-centric event Microsoft Ignite!
As always, I’m available on Twitter (@lorrainebardeen) and eager to hear about what you’re doing with mixed reality.
Talk soon!Lorraine
Updated July 31, 2018 2:56 pm

Mixed Reality @ Microsoft – June 2018 Update – Windows Experience Blog

Recent Microsoft-Harvard Business Review survey shows 87 percent of respondents are currently exploring, piloting, or deploying mixed reality in their company.
Hey everyone — I hope this month’s blog post finds you well!
Today, we are welcoming the solstice in the U.S., and I am very much looking forward to summer in Seattle. In addition to some planned vacation time, I will also be working with our team and partners on some exciting product development efforts for mixed-reality business applications. I can’t wait to share more about that in the coming months!
But before we look too far ahead, June has already been filled with some cool mixed-reality moments.
Earlier this month my colleagues Dio Gonzalez and Katie Kelly presented at the sixth annual Augmented World Expo (AWE) in Santa Clara, California. I was encouraged but not at all surprised to hear from them about the tremendous growth of the conference, with many more incredible and varied AR solutions than ever before. This mirrors the signals we’ve long observed at Microsoft and aligns with the level of activity we continue to see in this space: Mixed-reality technology is increasingly providing demonstrable value across a wide range of workplace scenarios, which is fueling further interest from developers and businesses alike. AWE is a great conference, and I hope to be able to join again next year.
Supporting this observation, Microsoft recently partnered with Harvard Business Review Analytic Services to conduct a survey investigating the unique role and importance of mixed reality within the context of the modern workplace. This research surveyed 394 executives of companies with more than 250 employees each and spanning several industries, from manufacturing, engineering, and construction to retail, defense, and education.
The results—which you can read here—were released today, and the findings are fascinating: Among a great many observations, we learned that 87 percent of respondents are currently exploring, piloting, or deploying mixed reality in their company work flows. Similarly, 68 percent of respondents believe that mixed reality will play an important role in helping to achieve their companies’ strategic goals over the next 18 months.
The survey results identified several exciting areas of opportunity in the growing mixed-reality space.
One of the key opportunities is with Firstline Workers, who make up 80 percent of the workforce but often have limited access to relevant, contextual information due to the on-the-field nature of their jobs. These are the workers who are typically on the frontlines of any business workflow: behind the counters, in the clinics, traveling between customers for field service, or on the factory floors. Several of Microsoft’s commercial customers, for instance, are already empowering their Firstline Workers today with mixed-reality solutions that enable remote assistance, spatial planning, environmentally contextual data, and much more. Mixed reality allows these Firstline Workers to conduct their usual, day-to-day activities with the added benefit of heads-up, hands-free access to incredibly valuable, contextual information.
Lastly, a couple of days ago Alex Kipman spoke about mixed reality in the modern workplace at LiveWorx in Boston. LiveWorx brings together BDMs, engineers, and developers to learn about the tools available to help drive digital transformation in the workplace – such as IoT, mixed reality, and robotics.
Given our mission to help empower people and companies to achieve more, the conference was a great fit for our team. Alex hit on Microsoft’s strategy for mixed reality, in particular how it will serve to accelerate our ambition for an Intelligent Cloud and an Intelligent Edge. For those who have been with us on our mixed-reality journey, and for those who are just joining us, his fireside chat with Jon Fortt is a must-watch.
I am already looking forward to next month’s blog. In the meantime, as always, I’m available on Twitter (@lorrainebardeen) and eager to hear about what you’re doing with mixed reality.
Talk soon!

Microsoft Remote Assist and Microsoft Layout available today as limited-time free preview

Hey everyone!
Wow, time really flies. I can’t believe it’s been two weeks since I talked about our vision for mixed reality in the modern workplace and unveiled Microsoft Remote Assist and Microsoft Layout at Build! That was such a fun day for me and my team. The feedback from customers, partners, and developers has been tremendous, and we can’t wait to deliver these apps to you all.
Fast forward to today, and I had the privilege of speaking at Tokyo de:code. I talked more about our vision for mixed reality at work, the changing modern workplace, and how products like HoloLens can empower both Information and Firstline Workers to achieve more. The positive energy and support coming out of Asia for mixed reality and Microsoft HoloLens has been overwhelming (more on that below).
As part of my presentation I made a couple of very important mixed-reality announcements that I wanted to share with you all.
Microsoft Remote Assist and Microsoft Layout available today as limited-time free preview
I love being able to change from “coming soon” to “available today!” The team has worked hard to get these ready for all of you to try out and give us feedback. Feedback will enable us to make them generally available to customers as soon as possible.
Remote Assist and Layout are available starting TODAY as a limited-time free preview in the Microsoft Store!
With Remote Assist we set out to create a HoloLens app that would help our customers collaborate remotely with heads-up, hands-free video calling, image sharing, and mixed-reality annotations.
With Layout our goal was to build an app that would help people use HoloLens to bring space designs from concept to completion much more quickly by viewing their designs at real-world size and scale.
A great way to see Remote Assist and Layout in action is through the work of ZF Group, a German car-parts manufacturer headquartered in Friedrichshafen. An early partner on our journey, ZF Group has been working with us over the past few months to help ensure these apps, even in preview, are crafted with insights from those who will be using them daily to get their work done. In developing these apps with customers, we are not only looking to make great software—we’re looking to empower the Firstline Workers using the apps.
Here’s a look into how ZF Group has been using Remote Assist and Layout on its factory floor.

Additional information, such as device requirements, technical requirements, and network requirements for both Remote Assist and Layout, can be found here.
Windows 10 April 2018 Update for Microsoft HoloLens coming soon
Second, I’m excited to announce that the Windows 10 April 2018 Update is coming to HoloLens! 
HoloLens customers in our Windows Insider Program have been testing previews of this release and have given us great feedback on some new features that will be generally available in the next few weeks.
There are quite a few important updates coming for IT administrators, developers, and our commercial customers.
For IT administrators
The Windows 10 April 2018 Update will improve the ability for IT administrators to provision, secure, manage, and update all HoloLens devices in their organization, including scenarios where multiple people use the same device.
This update provides improved support for Unified Endpoint Management solutions (also known as MDM) like Microsoft Intune, making it possible for IT admins to simplify and automate device management. Microsoft Intune, powered by the intelligent cloud, empowers employees to create and work together, securely, and enables organizations to protect corporate data in today’s diverse and complex mobile environment.
To integrate HoloLens solutions with different environments and platforms, we have also announced partnerships with additional Unified Endpoint Management providers including IBM MaaS360, MobileIron, and SOTI which will start delivering solutions to market soon.
For developers
For developers, the Windows 10 April 2018 Update will bring improvements in spatial mapping, automatic selection of focus point to improve hologram stability, new holographic projection modes, and APIs to help developers tailor apps.
It will also deliver a feature that I’m particularly excited about: research mode. Research mode allows developers to access key HoloLens sensors when building academic and industrial applications to test new ideas in the fields of computer vision and robotics. I can’t wait to see what our community comes up with using research mode!
For end customers
Last are the improvements we are bringing with this update that will directly benefit our customers who are using and sharing with HoloLens.
The Windows 10 April 2018 Update for HoloLens uses AI to implement features like auto-placement of new content (2-D and 3-D) on launch so you can seamlessly enter mixed-reality experiences. You’ll notice a new ability to fluidly manipulate apps when you want to move, resize, and rotate 2-D or 3-D content without having to enter “adjust” mode. You can make a 2-D UWP app like Mail wider to see more content. With expanded voice command support you can do more, simply using your voice. We’ve updated apps like Holograms and Photos and improved mixed-reality capture with a hardware shortcut.  And thanks to many requests, now there is support for File Manager and Media Transport Protocol, so you can connect HoloLens to your PC and easily transfer files.
Finally, we’ve simplified the space management for holograms and improved the immersive spatial audio. You can now hear HoloLens better in noisy environments, and experience more lifelike sound from applications because their sound will be obscured by real walls detected by the device.
There’s a whole lot more in this update as well. For a detailed description of the new capabilities, check out our Windows 10 April 2018 Update page.
Momentum for HoloLens and Mixed Reality in China and Japan
In addition to speaking at Tokyo de:code I have spent the past several days meeting with customers in Beijing and Tokyo. There is nothing I love more about my job than spending time with our customers.

On Friday, May 18, we celebrated the one-year anniversary of HoloLens in China. The growth we have experienced in just 12 months has been tremendous, a testament to the passion of the China mixed- reality development community—some of the most dynamic developers in the world.
Developers and partners have shown a tremendous willingness to transform their businesses in MR. We are seeing innovation in healthcare, education, government, AEC (Architecture, Engineering & Construction), manufacturing/auto, aviation, and media.
A few of the partners I spent time with included:
Arcplus – Arcplus is a leader in China’s AEC industry. Through its subsidiary Actron, it is leading digital transformation for construction and engineering. Arcplus believes that mixed reality, HoloLens, and Azure are key elements of its digital transformation strategy.
Visual3D – As part of our global Mixed Reality Partner Program, Visual3D has moved quickly to impact innovation in clinical medicine, medical teaching, and medical research through Microsoft HoloLens. Visual3D collaborated with Dr. Zhewei Ye of Wuhan Union Hospital to perform the first HoloLens-assisted remote cooperative orthopedic surgery.
DataMesh – Another member of our Mixed Reality Partner Program, DataMesh brings together Azure, HoloLens, and Mixed Reality, as well as data science to provide solutions for Firstline Worker scenarios. Through a collaboration with GAC Honda, DataMesh has created a digital twin of a Honda factory to show modern production processes.
In Japan it was more of the same. The passion, ingenuity, and creativity we see from customers, partners, and developers is mind-blowing. From the moment we brought HoloLens to Japan they have been setting an incredibly high bar. They have taken a bold approach toward implementing mixed reality that we believe will serve as a catalyst for invention others will surely follow.
I saw so many amazing use cases. Some of my favorites included:
Informatix – Tokyu Construction has partnered with Informatix, an MRPP partner, to create the application GyroEye Holo. I had a chance to see this in person at one of its construction sites—it was amazing to see actual blueprints layered over the real world.
JRCS – I’ve had the opportunity before to see the impressive work JRCS is doing to prepare for a future of autonomous shipping, to support their employees with high-quality training, and to enable more efficient remote maintenance and repair. That said, it was even better to meet the team themselves and try out a demo.
Pocket Queries – I had the chance to see the solution “QuantuMR” created by Pocket Queries in partnership with Tokyo Electric Power Company Holdings. This was an impressive four-step scenario using multiple building systems, IoT integration, and even a holographic printer!
I look forward to sharing more updates next month. As always, feel free to reach out to me on Twitter to share what you are doing with mixed reality.
Talk soon!

Introducing two new mixed reality business applications: Microsoft Remote Assist and Microsoft Layout

Hey everyone!
I am quite excited to share today’s blog. This morning I will be joining Satya Nadella at day one of Microsoft Build 2018 to announce two new apps coming to Microsoft HoloLens customers as a limited-time free preview starting May 22 — Microsoft Remote Assist and Microsoft Layout.
Last year I had the chance to speak at Future Decoded and talk about our vision for mixed reality in the modern workplace. In particular, I talked about expanding our focus from Information Workers to include Firstline Workers. Firstline Workers in any company are the first to the customer and the first to the problem. This includes people who primarily interact with customers, as well as people who create and operate products, machines, and spaces and who need to move around and use their hands to do their work.
We learn so much from others who are using what we create early on. Customer feedback is our most important development tool. Over the past two years since we started shipping HoloLens, we’ve been paying attention to the highest value workflows that cut across a variety of Firstline industries. In close partnership with hundreds of customers, we discovered that the highest value workflows are around remote assistance, space planning, training and development, product development collaboration, and access to spatial data from IoT devices.
Over the past several months, our team has been working hard to develop and deliver applications that will empower Firstline Workers to achieve more. It’s my privilege on behalf of everyone on our team to introduce you to Microsoft Remote Assist and Microsoft Layout, coming as a limited-time free preview for existing HoloLens customers starting May 22.
To learn more about how you can participate in the limited-time free preview starting May 22, please visit us here.
Microsoft Remote Assist — Collaborate in mixed reality to solve problems faster
With Microsoft Remote Assist we set out to create a HoloLens app that would help our customers collaborate remotely with heads-up, hands-free video calling, image sharing, and mixed-reality annotations. During the design process, we spent a lot of time with Firstline Workers. We asked ourselves, “How can we help Firstline Workers share what they see with an expert while staying hands-on to solve problems and complete tasks together, faster.” It was important to us that Firstline Workers are able to reach experts on whatever device they are using at the time, including PCs, phones, or tablets.
Our work led us to focus on three capabilities.
Collaborate with shared perspective
With video calling, made possible using Microsoft Teams, image sharing, and mixed-reality annotations, Firstline Workers can share what they see and collaborate with experts remotely. Workers and experts can annotate their shared view with mixed-reality ink and arrows, or insert images into their view, to pinpoint and solve problems efficiently.
Communicate securely
The ability to control access to remote communications with industry-leading identity and security measures. With Azure Active Directory login and Mobile Device Management, Firstline Workers and experts can focus with peace of mind on what matters most.
Increase efficiency
With mixed-reality annotations, live streaming, and video capture, we can enable Firstline Workers and experts to identify and address issues accurately the first time. This can help customers eliminate travel and expedite troubleshooting, increasing employees’ efficiency.
Here is a peek into what Microsoft Remote Assist will make possible.

Microsoft Layout — Design spaces in context with mixed reality
With Microsoft Layout our goal was to build an app that would help people use HoloLens to bring designs from concept to completion using some of the superpowers mixed reality makes possible. With Microsoft Layout customers can import 3-D models to easily create and edit room layouts in real-world scale. Further, you can experience designs as high-quality holograms in physical space or in virtual reality and share and edit with stakeholders in real time.
With Microsoft Layout we wanted to help customers see ideas in context, saving valuable time and money. Here is a peek into what Microsoft Layout makes possible.

We want to make sure you have what you need to prepare for the limited-time free preview of Remote Assist and Layout when they are released on May 22. To make sure your roll-out is smooth and seamless, additional information, such as device requirements, technical requirements, and network requirements can be found here.
As excited as we are about these apps, there is nothing more important than seeing how our customers and partners are using them. The workplace is changing now more than ever. While we live in a world where technology is everywhere, many people don’t have information where they need it most: in the real-world context of their work.
Today, as part of our presentation I was able to showcase how ZF Group, a German car-parts manufacturer headquartered in Friedrichshafen, is using Microsoft Remote Assist and Microsoft Layout on its factory floor. An early partner on our journey, it has been working with us over the past few months to help ensure these apps, even in preview, are crafted with insights from those who will be using them daily to get their work done.
Here’s a look into how ZF Group has been using Remote Assist and Layout over the past couple of months.

World-sensing devices will use spatial analytics to make data more valuable
Finally, we talked about the empowerment that comes from world-sensing devices that can provide invaluable spatial data and context for physical workflows that matter most. These environments are an untapped opportunity for getting work done more effectively. We’re taking this further by making sure this class of data will have enterprise-grade manageability and extensibility that integrates with existing applications and processes.
One of our earliest HoloLens partners, thyssenkrupp, is leveraging this work now to discover insights that will make it easier for its employees to complete some of their most important tasks.
Here is a look into the work our friends at thyssenkrupp are doing to leverage spatial analytics.

We will have a lot more to share in the coming weeks and months. Soon you will hear us share more on how we are going to help with training and development and collaboration on product design and prototyping. Our team loves doing work that really makes a difference for our customers!
And a reminder — if you want to be a part of the preview for Microsoft Remote Assist and Microsoft Layout, please visit us here.
As always, feel free to reach out to me on Twitter to share what you are doing with mixed reality.
Talk soon!

Introducing two new mixed reality business applications: Microsoft Remote Assist and Microsoft Layout
Tweet This

Mixed reality in manufacturing comes to life at Hannover Messe

I hope this month’s mixed reality blog post finds you well!
For this entry, I am going to focus on some of the amazing work we are seeing from customers and partners this week at the Hannover Messe (HM) show in Germany.
Last year, we highlighted some exciting customer stories, including how thyssenkrupp is transforming the delivery of home mobility solutions using Microsoft HoloLens.
This year, we are demonstrating how HoloLens and Windows Mixed Reality are helping companies reimagine processes, data, and people to accelerate business impact and create value for customers.
The manufacturing industry has been aggressively adopting new technologies that empower Firstline Workers to solve problems, achieve efficiencies, and better serve customers. Mixed reality is bringing these benefits to life, and today, manufacturers around the world are realizing cost savings and efficiency gains as a result.
Here is a view into some of the mixed reality solutions on display at the Microsoft booth this week at Hannover Messe
BAE Systems and PTC
BAE Systems makes the electric propulsion Systems for HybriDrive® buses. BAE is working with PTC, a Microsoft Mixed Reality Partner, utilizing their ThingWorx Studio to create mixed reality solutions that dramatically improve the efficiency of Firstline Workers. BAE and PTC used ThingWorx Studio to easily create a guided step-by-step training solution for HoloLens to teach workers how to assemble a green energy bus battery. Using these tools, BAE can now create these guides for firstline workers in just a few hours at a tenth of the cost, training new people 30-40% more efficiently.
Here is a look at how BAE Systems and PTC are using HoloLens:

Honeywell Maintenance Training Simulator
In February, our friends at Honeywell introduced the Honeywell Maintenance Training Simulator. This work empowers system maintenance personnel to be prepared for a variety of critical situations and to address them faster with reduced risk. It enables their employees to test and enhance their knowledge in a safe and simulated immersive environment prior to managing critical infrastructure.
Mercedes-Benz Global Training
Mercedes-Benz Microsoft HoloLens brake assembly
Mercedes-Benz Global Training is showing off how HoloLens can be used to improve the efficiency of brake assembly and engine manufacturing. In addition, they are demonstrating how customers can load a roof box with their virtual bags or sports items, such as skis and equipment in a real roof box mounted on an E-Class sedan.
Autodesk Project Sugarhill
Autodesk is demonstrating Project Sugarhill, a technical preview of immersive conceptual design tools that empower artists and designers to start their creative process directly in 3D. Using simple curve and surfacing tools, users can explore form and shape while being fully immersed alongside their design. 3D sketches and models can be easily exported to other screen-based applications such as Alias or Maya for final realization.
Deutsche Bahn
Deutsche Bahn railway training in mixed reality
Deutsche Bahn has worked with the Microsoft Mixed Reality Partner Program to build an app that will enable the German railway company to more efficiently and safely train engineers on railway parts and maintenance. Really cool to see them bring this to life on the show floor!
Microsoft Visio
This week, we’re delighted to demonstrate Microsoft Visio updates designed to address intelligent manufacturing scenarios, including facility layout. As part of our work at HM, we will be showcasing Intelligent Manufacturing scenarios supported by Microsoft customer Dürr AG and incorporating Microsoft Visio Partners FaciWare GmbH (facility management solutions) and X-Visual Technologies GmbH (PnID solutions).
It’s been exciting to watch mixed reality solutions for the manufacturing industry transition from proof-of-concept to production deployments that deliver tangible benefits to manufacturers. We remain inspired by the ways our customers and partners are using mixed reality and HoloLens to do things that were previously impractical or impossible.
I can’t wait to share more in the weeks and months ahead!
As always, feel free to reach out to me on Twitter @lorrainebardeen to share what you are doing with mixed reality.
Talk soon!

Mixed reality in manufacturing comes to life at Hannover Messe
Tweet This

Windows 10 Tip: 10 things you can do in Windows Mixed Reality right now

To experience Windows Mixed Reality, you’ll need a few things: a compatible Windows 10 PC, a Windows Mixed Reality headset, and the Windows 10 Fall Creators Update for your PC. For the best experience, you’ll also want a pair of Windows Mixed Reality motion controllers.

Now, all set up but not sure where to begin? Here’s a list of ten things for you to try in Windows Mixed Reality right now.

Personalize your display

Before you start playing in mixed reality, you’ll want to adjust your headset display for your interpupillary distance (IPD). Since everyone has a different distance between their eyes, it’s important to change your headset’s software settings to match your own IPD for the best image quality and depth accuracy when you play. You can set your custom IPD via Settings > Mixed Reality > Headset display > Calibration.[1]

Customize your mixed reality home

The first thing you’ll see when you launch Windows Mixed Reality and put on your headset is your mixed reality home, your home base for discovering specific apps or experiences. Here, you can customize each room to your liking. Open the Start menu to pin your favorite apps to the walls and go to Start > Holograms to add furniture, people, and other holograms to your space.

Navigate in mixed reality

There are several ways to get around in mixed reality with your motion controllers. To teleport, point your controller to where you want to go, press either thumbstick forward, and release to instantly land there. To rotate, press your thumbstick left or right, and to back up, press it back. To walk continuously, press either thumbstick straight down and move it in the direction you want to go.

Use Cortana in mixed reality

Did you know you can use Cortana while in Windows Mixed Reality? Cortana can help you get around faster with your voice. For example, you can have Cortana adjust the volume in a game, open an app, and even teleport without a controller. Simply start with “Hey Cortana,” to try it now.[2]

Play games in Steam®VR

Did you know that in addition to your favorite apps from the Microsoft Store, you can access more than 2,500 amazing games and VR experiences through Steam®VR for Windows Mixed Reality? [3] To get set up, go to on your PC. Once you’ve launched Steam®VR from your headset, use your motion controllers and press straight down on the left or right thumbstick to open the dashboard and start exploring.

Watch immersive videos on the web

Immerse yourself in 360° videos and photos on the web. Get the 360 Viewer extension for Microsoft Edge and put on your headset to browse immersive content from YouTube, Facebook,, and more.

Take a closer look in mixed reality

For a closer look at any open app, you can zoom with your motion controllers. To zoom in, point at an app window with both controllers, pull both triggers, and move your hands apart. To zoom back out, bring your hands together.

View apps from the best angle

To perfectly center yourself in front of an app, bounce your teleport off the open app window. To do this, press your thumbstick forward, aim the controller at the app you want to use, and release the thumbstick to land exactly in front of the app window.

Adjust the floor of your mixed reality home

Sometimes, the floor of your Windows Mixed Reality home may feel slanted or at the wrong height. If it doesn’t feel comfortable to you, you can change the floor height via Start > Room Adjustment. Just follow the instructions and use the touch pad on your motion controller to get your floor feeling right in no time.

Share your experience with others

Use the Mixed Reality Portal to share the fun with friends. From your compatible PC, select the Play button to show the view from your headset on your computer screen.3
These are just a few of the many things you can do in Windows Mixed Reality. To learn more, check out  Windows Mixed Reality Support and the mixed reality tips page for even more inspiration.
[1] In addition, the Samsung HMD Odyssey headset has a mechanical IPD adjuster. [2] To experience sound, Cortana and voice dictation, compatible mic-enabled headphones with 3.5mm jack (USB won’t be able to connect to headset) have to be attached, plugged in to the audio jack on the HMD. Consumers may find compatible headphones by looking for the Cortana badge or Circle icon on product packaging and websites. [3] PC hardware requirements may vary for available apps, features and content.

Computer Aided Design (CAD) enters the era of mixed reality

I’m excited to share this month’s blog post with you all. In addition to highlighting some of the cool work we are seeing from the mixed reality community, I also have some important Simplygon news to share.
In the time since we acquired Simplygon they have been working quickly to bring the value of the Simplygon service to mixed reality.
Let’s dig in!
Bringing Computer Aided Design (CAD) into the era of mixed reality
Throughout history, media portability has been critical to the adoption of new computing experiences. For example, prior to the mid-1990s, compact disc owners were unable to move their music from physical to digital media. As audio encoding technology became ubiquitous, music and other digital files became much easier to move between devices. This portability enabled the journey from physical media to mobile mp3 players, to digital music stores, and finally to today’s world where access to nearly every song lies just a few clicks away.
We think about examples like this as we make investments to support the extensibility of 3D content across screens and into mixed reality. 3D content of all types – static, animated, simple, and complex – is key to transformative mixed reality experiences. This is why we’ve invested to support GL Transmission Format (glTF) (known by many as “JPEG of 3D”) as an industry standard 3D file format and to bring Simplygon’s 3D asset optimization and decimation services into the Microsoft family.
Today, I am excited to announce that Simplygon has added compatibility for JT and STEP file formats, which are pivotal to the productivity of CAD professionals. The addition of two of the most common CAD file formats brings automation of 3D optimization to customers across many industries. As the enterprise increasingly embraces the era of digital transformation, CAD support for Simplygon empowers more industries to painlessly and efficiently bring their existing assets into mixed reality.
For 3D content, this announcement is another early step along that same transformative path. CAD users are now able to create on their PC in JT or STEP formats, transcode and optimize via Simplygon, and distribute their work to their preferred device endpoint. We will continue taking steps along the journey to 3D portability and making the proliferation of 3D content as simple as possible.
JT and STEP are two of the most common CAD file types and are both ISO standards. These file types are heavily used in manufacturing industries and are vital to solving 3D problems. JT files, used heavily in product lifecycle management software, is primarily used to analyze and augment the geometry of complex models. STEP, or Standard for the Exchange of Product model data, is primarily used to share 3D models between users with different CAD software packages.
How to get started
Simplygon Cloud is available in Azure Marketplace today. To start, visit our Azure Marketplace home to learn how to deploy the Simplygon Cloud virtual machine and start optimizing your 3D assets. Please also visit our documentation for examples and more information on how to integrate this into your workflow today.
Continued adoption of mixed reality
During the month of March, we saw some fun, unique, and inspiring work from the global mixed reality community. Here were some of my favorites!
Toms River Municipal Utilities Authority (TRMUA) uses mixed reality to view simulated GIS features
This is so cool! Len Bundra, the IT/GIS director for the Toms River Municipal Utilities Authority (TRMUA) in Toms River, New Jersey and Alec Pestov, a developer from Toronto, Canada and Founder and CEO of Meemim worked together to create a mixed reality experience that would allow utilities (and eventually other organizations) to collaborate with shared content and data. When we talk about mixed reality and HoloLens giving people super-powers that change the way we work, this is exactly the type of innovation we are talking about!

Renault Sport Formula One team brings their creations to life with Microsoft HoloLens
I loved reading about the work that the Renault Sport Formula One Team is doing with HoloLens to make it easier for race fans to experience their one of a kind (literally!) creations. As Mark Everest, IS Development Manager at Renault Sport Formula One Team notes, “We build two cars a year, not thousands of cars. The innovation rate involved in design, simulation, testing and manufacturing is much faster than consumer car companies.”
Before each new season begins, an entirely new car is born. The Renault Sport Formula One Team has partnered with Microsoft to create a mixed reality experience that brings the car to life at events around the world – without the car (since there is only one) physically having to be there. And while the team is exploring how the HoloLens could impact engineering in the future, this experience will immerse viewers in the current evolution of design from earlier cars — like the RE40 (1983), R26 (2006), and R31 (2011) — to the most recent models, paired with facts about the design process and performance.

Reinventing Brian Eno and Peter Chilvers’ ‘Bloom’ for mixed rality
Musician, producer, visual artist, and thinker Brian Eno, together with longtime collaborator Peter Chilvers, launched their first mixed reality project with Bloom: Open Space, a generative audio-visual installation brought to life with HoloLens. Based on their award-winning app, Bloom, the innovative mixed reality installation blurs the lines between the physical and virtual, exploring uncharted territory in the realms of both applications and generative art.

Murmur: Arctic Realities at Mystic Seaport
I have written before about how inspired I am when people use mixed reality to tell stories and paint an otherwise blank canvas. This month I wanted to call attention to an exhibit that is coming to life at Mystic Seaport in New Haven, CT.
Murmur: Arctic Realities explores Arctic landforms called pingos – ice hills rising from permafrost soil, covered by a thin layer of earth. These hills grow over centuries and then collapse, pockmarking the landscape in the Circumpolar North.
Seattle artist John Grade and new media artist Reilly Donovan have partnered to present an immersive technological component for Murmur using HoloLens. Through HoloLens, Grade and Donovan have created a holographic patchwork of tundra, complete with water, living organisms, and the sound of Arctic birds. As visitors move about the gallery they will notice that aspects of the experience respond to their presence.

I look forward to sharing our update for next month where we will dive into the work we are doing along with our customers and partners at Hannover Messe, one of my many favorite industry events.
Talk soon!
(P.S. – feel free to follow me on Twitter @lorrainebardeen where I regularly share updates and answer questions from the mixed reality community)

Computer Aided Design (CAD) enters the era of mixed reality
Tweet This

Windows 10 Tip: View 360° videos and photos in Microsoft Edge with your Windows Mixed Reality headset

With the 360 Viewer extension and a Windows Mixed Reality headset plugged into your PC, you can view 360° videos and photos in your headset from Microsoft Edge, simply by clicking the Windows Mixed Reality icon that has been added to the video or photo.
You can also navigate to a website in Microsoft Edge on your desktop, play a 360° video and click on the Mixed Reality icon. This will automatically launch Windows Mixed Reality and start playing the 360° video in your headset, as long as you have your headset plugged into your PC.
Once you’ve downloaded this extension, you can browse and see 360° photos, watch 360° YouTube videos of a shark encounter or enjoy 360° videos from the NYTimes website – all in your Windows Mixed Reality headset from Microsoft Edge.
To learn more about Windows Mixed Reality, head over here!
*Currently supports content from YouTube, Facebook, Twitter, and NYTimes. Requires a Windows Mixed Reality immersive headset and compatible PC.

How mixed reality is changing the game for healthcare, from performing live surgeries to delivering ultrasounds in 3D

Last year around this time I had the chance to write a post about one of my favorite mixed reality uses cases – healthcare.
Since that time, we have continued to see remarkable momentum for mixed reality across numerous facets of the industry. This week at HIMSS our team had the privilege to once again be on the ground to see and experience how customers and partners are embracing mixed reality to transform the way they innovate, support, and help others.
Coming out of the show I wanted to take a moment to spotlight some of the innovation work we are seeing from organizations who are working to forge the future of healthcare.
CAE Healthcare
Joining us onsite at HIMSS this year was CAE Healthcare. CAE is an industry leader in the development of simulation-based technologies, curriculum, and resources that improve clinical performance. I talked about some of their earliest work with Microsoft HoloLens last year – well, they certainly haven’t slowed down!
CAE VimedixAR is the first ultrasound simulator to integrate HoloLens. Freed from the limits of a two-dimensional environment inside a monitor, healthcare professionals can display, enlarge, turn, and rotate realistic-looking anatomical parts, or command them to return into the manikin body. Users and those learning with this technology are able to witness (in real time) the ultrasound beam as it cuts through human anatomy.

With CAE LucinaAR, clinical learners can view 3D holograms of a fetus as it descends the birth canal and gain an unprecedented view of anatomy as they learn to manage a shoulder dystocia delivery. CAE Lucina is a wireless childbirth simulator with validated, integrated maternal-fetal physiology for training on all the stages of delivery and the rare emergency scenario. With five pre-configured Simulated Clinical Experiences (SCEs), and new mixed reality modules coming soon, Lucina allows instructors at universities and hospitals to easily integrate simulation and augmented reality into their hands-on educational programs.

With their Abiomed Impella procedural training solution, CAE Healthcare is integrating HoloLens into its simulation-based training platform that allows physicians to practice ultrasound-guided placement of the world’s smallest heart pump. Overlaying the ultrasound manikin, probe and instruments, the CAE HoloLens framework will depict physiological responses and potential complications to help clinicians quickly master and deploy the device without risk to patients.
Adding to its physics-driven simulator for the Medtronic Micra Transcatheter Pacing System (TPS), the world’s smallest pacemaker, CAE Healthcare delivered a mixed reality training solution for up to 11 simultaneous learners on its Microsoft HoloLens platform. Learners can practice while viewing interactive holograms of a 3D beating heart into which the Micra delivery system is visible and connected in real-time.
Pearson delivers mixed reality learning tools for nursing professionals
At Bett this year we were thrilled to announce some of the great curricula our friends at Pearson were delivering to train nurses using mixed reality and the benefit of holographic patients. Today, I am happy to share more details about one of their extraordinary pieces of the curriculum – HoloPatient.
HoloPatient is a mixed-reality learning tool for nursing, allied health and medical schools that will deliver simulated patient experiences to more students, at a more manageable cost, and with a greater degree of consistency than ever before. This cutting-edge 3D immersive solution redefines how learners interact with their worlds and each other—bringing learning to life with startling clarity and breathing new possibility into teaching. Designed and conceived in collaboration with expert healthcare educators and simulation facilitators from around the world, HoloPatient uses Microsoft’s Mixed Reality Capture technology allowing tutors and students to engage in a range of learning and assessment activities previously only possible when using actors or by encountering patients in real life.

HoloPatient drives consistency across the delivery of learning content and assessment while also allowing nursing and medical schools to augment their existing delivery of world-class education and training with one of the most promising and engaging teaching and learning use-cases for mixed reality currently available.
The HoloPatient app will be released globally in April 2018, along with a virtual reality version later in the year. By the end of June 2018, Pearson will have released four other learning tools including Anatomy, Chemistry (building molecules in 3D), Math (Area and Volume of 3D objects) and History.
You can find out more at
SphereGen is utilizing mixed reality to create innovative solutions designed to increase efficiency and revolutionize the learning process. SphereGen collaborated with St. George’s University to develop the Learning Heart. The Learning Heart is a HoloLens application that assists students in understanding the elements of the Heart. As a study aid and teaching tool, this application can be used individually or collaboratively by multiple HoloLens users. The holograms are fully three-dimensional and can be traversed around and viewed from any position and angle.

DICOM Director is a full product suite designed to enable communication and collaboration between different radiologists, doctors, and medical practices. The backend of this solution enables radiologists to view, transfer and share scans. This solution utilizes HoloLens as a viewer for radiological scans, such as MRIs, cat scans, and several others. Radiologists and doctors can remotely view their scans through the HoloLens and/or a Windows Mixed Reality headset. The viewing functionality enables the user to see the series of scans, an enlarged view of the selected scan (which they can scroll through), and the 3D model reconstruction from the scans.

Live surgery with HoloLens
One of the most mind-blowing moments in mixed reality occurred just a few months ago. On that day AP-HP, Paris French hospitals group, conducted a live surgery on YouTube, assisted by TeraRecon medical holographic platform leveraging Azure GPU and HoloLens. The surgeon was assisted by TeraRecon and their partners to develop the complete end-to-end solution from the patient surgery planning up to the real-time streaming of the real patent image data (pre-operative 3D CT scan) with its full volume rendering in the highest quality ever seen including segmentation of the CT scan. This live surgery was conduct using sharing experience to share the same patient data across multiple HoloLens with the same holographic space (operating room) as well as sharing the HoloLens view with the remote surgeon.
With a mission to empower doctors, Visual 3D Medical Science and Technology Development CO. LLC) is dedicated to the use of artificial intelligence, cloud computing, and mixed reality to enhance the capability of doctors. By way of research and development, Visual3D’s technology can promote doctor’s ability of perception and appliance of medical information; enhance the remote cooperative abilities of doctors in different places to build a scientific and technological platform for intelligent medical therapy.
Visual3D has already performed over 200 hip, knee, and spinal surgeries in China using HoloLens. In addition to using HoloLens in the operating theater, Visual3D is using HoloLens to assist doctors pre-surgery as both a tool to brief patients on their upcoming surgical procedures, as well as allowing doctors to pre-visualize and rehearse the procedure before entering the operating room.
Next Surgeries
Digital Pages, a leading Brazilian company in the lifecycle management of digital contents, has developed an interactive application for surgery planning using Microsoft HoloLens. The application offers students, surgeons and healthcare professionals a new perspective on the modeling and manipulation of physical and virtual 3D objects. Next Surgeries introduces the use of holographic projections in surgical planning, reducing the total time of surgeries, the exposure of the patient and staff to radiation and risk of infections, blood loss, etc. The application is available on the digital teaching platform RDP Learning integrated to HoloLens, bringing innovation and more security to the health sector.
I look forward to sharing more with you all soon as we continue to shine a light on many of the great mixed reality innovations being delivered by customers, partners, and developers of mixed reality!

#ifdef WINDOWS – How to enable WebVR with just two lines of code with BabylonJS

BabylonJS is a very powerful JavaScript framework for building 3D apps and games with web standards, used by game developers to build some amazing experiences that can run on any platform and device. This includes Windows Mixed Reality and VR platforms such as Oculus or SteamVR.  
With the latest release of BabylonJS, developers can enable immersive experiences with only 2 lines of code. David Rousset, one of the core authors of BabylonJS, stopped by to show just how easy it is to create fully interactive WebVR apps. Watch the full video above and feel free to reach out on Twitter or in the comments below for questions or comments. 
Happy coding!