Category Archives: Microsoft HoloLens

Auto Added by WPeMatico

Microsoft Remote Assist and Microsoft Layout available today as limited-time free preview

Hey everyone!
Wow, time really flies. I can’t believe it’s been two weeks since I talked about our vision for mixed reality in the modern workplace and unveiled Microsoft Remote Assist and Microsoft Layout at Build! That was such a fun day for me and my team. The feedback from customers, partners, and developers has been tremendous, and we can’t wait to deliver these apps to you all.
Fast forward to today, and I had the privilege of speaking at Tokyo de:code. I talked more about our vision for mixed reality at work, the changing modern workplace, and how products like HoloLens can empower both Information and Firstline Workers to achieve more. The positive energy and support coming out of Asia for mixed reality and Microsoft HoloLens has been overwhelming (more on that below).
As part of my presentation I made a couple of very important mixed-reality announcements that I wanted to share with you all.
Microsoft Remote Assist and Microsoft Layout available today as limited-time free preview
I love being able to change from “coming soon” to “available today!” The team has worked hard to get these ready for all of you to try out and give us feedback. Feedback will enable us to make them generally available to customers as soon as possible.
Remote Assist and Layout are available starting TODAY as a limited-time free preview in the Microsoft Store!
With Remote Assist we set out to create a HoloLens app that would help our customers collaborate remotely with heads-up, hands-free video calling, image sharing, and mixed-reality annotations.
With Layout our goal was to build an app that would help people use HoloLens to bring space designs from concept to completion much more quickly by viewing their designs at real-world size and scale.
A great way to see Remote Assist and Layout in action is through the work of ZF Group, a German car-parts manufacturer headquartered in Friedrichshafen. An early partner on our journey, ZF Group has been working with us over the past few months to help ensure these apps, even in preview, are crafted with insights from those who will be using them daily to get their work done. In developing these apps with customers, we are not only looking to make great software—we’re looking to empower the Firstline Workers using the apps.
Here’s a look into how ZF Group has been using Remote Assist and Layout on its factory floor.

Additional information, such as device requirements, technical requirements, and network requirements for both Remote Assist and Layout, can be found here.
Windows 10 April 2018 Update for Microsoft HoloLens coming soon
Second, I’m excited to announce that the Windows 10 April 2018 Update is coming to HoloLens! 
HoloLens customers in our Windows Insider Program have been testing previews of this release and have given us great feedback on some new features that will be generally available in the next few weeks.
There are quite a few important updates coming for IT administrators, developers, and our commercial customers.
For IT administrators
The Windows 10 April 2018 Update will improve the ability for IT administrators to provision, secure, manage, and update all HoloLens devices in their organization, including scenarios where multiple people use the same device.
This update provides improved support for Unified Endpoint Management solutions (also known as MDM) like Microsoft Intune, making it possible for IT admins to simplify and automate device management. Microsoft Intune, powered by the intelligent cloud, empowers employees to create and work together, securely, and enables organizations to protect corporate data in today’s diverse and complex mobile environment.
To integrate HoloLens solutions with different environments and platforms, we have also announced partnerships with additional Unified Endpoint Management providers including IBM MaaS360, MobileIron, and SOTI which will start delivering solutions to market soon.
For developers
For developers, the Windows 10 April 2018 Update will bring improvements in spatial mapping, automatic selection of focus point to improve hologram stability, new holographic projection modes, and APIs to help developers tailor apps.
It will also deliver a feature that I’m particularly excited about: research mode. Research mode allows developers to access key HoloLens sensors when building academic and industrial applications to test new ideas in the fields of computer vision and robotics. I can’t wait to see what our community comes up with using research mode!
For end customers
Last are the improvements we are bringing with this update that will directly benefit our customers who are using and sharing with HoloLens.
The Windows 10 April 2018 Update for HoloLens uses AI to implement features like auto-placement of new content (2-D and 3-D) on launch so you can seamlessly enter mixed-reality experiences. You’ll notice a new ability to fluidly manipulate apps when you want to move, resize, and rotate 2-D or 3-D content without having to enter “adjust” mode. You can make a 2-D UWP app like Mail wider to see more content. With expanded voice command support you can do more, simply using your voice. We’ve updated apps like Holograms and Photos and improved mixed-reality capture with a hardware shortcut.  And thanks to many requests, now there is support for File Manager and Media Transport Protocol, so you can connect HoloLens to your PC and easily transfer files.
Finally, we’ve simplified the space management for holograms and improved the immersive spatial audio. You can now hear HoloLens better in noisy environments, and experience more lifelike sound from applications because their sound will be obscured by real walls detected by the device.
There’s a whole lot more in this update as well. For a detailed description of the new capabilities, check out our Windows 10 April 2018 Update page.
Momentum for HoloLens and Mixed Reality in China and Japan
In addition to speaking at Tokyo de:code I have spent the past several days meeting with customers in Beijing and Tokyo. There is nothing I love more about my job than spending time with our customers.

On Friday, May 18, we celebrated the one-year anniversary of HoloLens in China. The growth we have experienced in just 12 months has been tremendous, a testament to the passion of the China mixed- reality development community—some of the most dynamic developers in the world.
Developers and partners have shown a tremendous willingness to transform their businesses in MR. We are seeing innovation in healthcare, education, government, AEC (Architecture, Engineering & Construction), manufacturing/auto, aviation, and media.
A few of the partners I spent time with included:
Arcplus – Arcplus is a leader in China’s AEC industry. Through its subsidiary Actron, it is leading digital transformation for construction and engineering. Arcplus believes that mixed reality, HoloLens, and Azure are key elements of its digital transformation strategy.
Visual3D – As part of our global Mixed Reality Partner Program, Visual3D has moved quickly to impact innovation in clinical medicine, medical teaching, and medical research through Microsoft HoloLens. Visual3D collaborated with Dr. Zhewei Ye of Wuhan Union Hospital to perform the first HoloLens-assisted remote cooperative orthopedic surgery.
DataMesh – Another member of our Mixed Reality Partner Program, DataMesh brings together Azure, HoloLens, and Mixed Reality, as well as data science to provide solutions for Firstline Worker scenarios. Through a collaboration with GAC Honda, DataMesh has created a digital twin of a Honda factory to show modern production processes.
In Japan it was more of the same. The passion, ingenuity, and creativity we see from customers, partners, and developers is mind-blowing. From the moment we brought HoloLens to Japan they have been setting an incredibly high bar. They have taken a bold approach toward implementing mixed reality that we believe will serve as a catalyst for invention others will surely follow.
I saw so many amazing use cases. Some of my favorites included:
Informatix – Tokyu Construction has partnered with Informatix, an MRPP partner, to create the application GyroEye Holo. I had a chance to see this in person at one of its construction sites—it was amazing to see actual blueprints layered over the real world.
JRCS – I’ve had the opportunity before to see the impressive work JRCS is doing to prepare for a future of autonomous shipping, to support their employees with high-quality training, and to enable more efficient remote maintenance and repair. That said, it was even better to meet the team themselves and try out a demo.
Pocket Queries – I had the chance to see the solution “QuantuMR” created by Pocket Queries in partnership with Tokyo Electric Power Company Holdings. This was an impressive four-step scenario using multiple building systems, IoT integration, and even a holographic printer!
I look forward to sharing more updates next month. As always, feel free to reach out to me on Twitter to share what you are doing with mixed reality.
Talk soon!

Introducing two new mixed reality business applications: Microsoft Remote Assist and Microsoft Layout

Hey everyone!
I am quite excited to share today’s blog. This morning I will be joining Satya Nadella at day one of Microsoft Build 2018 to announce two new apps coming to Microsoft HoloLens customers as a limited-time free preview starting May 22 — Microsoft Remote Assist and Microsoft Layout.
Last year I had the chance to speak at Future Decoded and talk about our vision for mixed reality in the modern workplace. In particular, I talked about expanding our focus from Information Workers to include Firstline Workers. Firstline Workers in any company are the first to the customer and the first to the problem. This includes people who primarily interact with customers, as well as people who create and operate products, machines, and spaces and who need to move around and use their hands to do their work.
We learn so much from others who are using what we create early on. Customer feedback is our most important development tool. Over the past two years since we started shipping HoloLens, we’ve been paying attention to the highest value workflows that cut across a variety of Firstline industries. In close partnership with hundreds of customers, we discovered that the highest value workflows are around remote assistance, space planning, training and development, product development collaboration, and access to spatial data from IoT devices.
Over the past several months, our team has been working hard to develop and deliver applications that will empower Firstline Workers to achieve more. It’s my privilege on behalf of everyone on our team to introduce you to Microsoft Remote Assist and Microsoft Layout, coming as a limited-time free preview for existing HoloLens customers starting May 22.
To learn more about how you can participate in the limited-time free preview starting May 22, please visit us here.
Microsoft Remote Assist — Collaborate in mixed reality to solve problems faster
With Microsoft Remote Assist we set out to create a HoloLens app that would help our customers collaborate remotely with heads-up, hands-free video calling, image sharing, and mixed-reality annotations. During the design process, we spent a lot of time with Firstline Workers. We asked ourselves, “How can we help Firstline Workers share what they see with an expert while staying hands-on to solve problems and complete tasks together, faster.” It was important to us that Firstline Workers are able to reach experts on whatever device they are using at the time, including PCs, phones, or tablets.
Our work led us to focus on three capabilities.
Collaborate with shared perspective
With video calling, made possible using Microsoft Teams, image sharing, and mixed-reality annotations, Firstline Workers can share what they see and collaborate with experts remotely. Workers and experts can annotate their shared view with mixed-reality ink and arrows, or insert images into their view, to pinpoint and solve problems efficiently.
Communicate securely
The ability to control access to remote communications with industry-leading identity and security measures. With Azure Active Directory login and Mobile Device Management, Firstline Workers and experts can focus with peace of mind on what matters most.
Increase efficiency
With mixed-reality annotations, live streaming, and video capture, we can enable Firstline Workers and experts to identify and address issues accurately the first time. This can help customers eliminate travel and expedite troubleshooting, increasing employees’ efficiency.
Here is a peek into what Microsoft Remote Assist will make possible.

Microsoft Layout — Design spaces in context with mixed reality
With Microsoft Layout our goal was to build an app that would help people use HoloLens to bring designs from concept to completion using some of the superpowers mixed reality makes possible. With Microsoft Layout customers can import 3-D models to easily create and edit room layouts in real-world scale. Further, you can experience designs as high-quality holograms in physical space or in virtual reality and share and edit with stakeholders in real time.
With Microsoft Layout we wanted to help customers see ideas in context, saving valuable time and money. Here is a peek into what Microsoft Layout makes possible.

We want to make sure you have what you need to prepare for the limited-time free preview of Remote Assist and Layout when they are released on May 22. To make sure your roll-out is smooth and seamless, additional information, such as device requirements, technical requirements, and network requirements can be found here.
As excited as we are about these apps, there is nothing more important than seeing how our customers and partners are using them. The workplace is changing now more than ever. While we live in a world where technology is everywhere, many people don’t have information where they need it most: in the real-world context of their work.
Today, as part of our presentation I was able to showcase how ZF Group, a German car-parts manufacturer headquartered in Friedrichshafen, is using Microsoft Remote Assist and Microsoft Layout on its factory floor. An early partner on our journey, it has been working with us over the past few months to help ensure these apps, even in preview, are crafted with insights from those who will be using them daily to get their work done.
Here’s a look into how ZF Group has been using Remote Assist and Layout over the past couple of months.

World-sensing devices will use spatial analytics to make data more valuable
Finally, we talked about the empowerment that comes from world-sensing devices that can provide invaluable spatial data and context for physical workflows that matter most. These environments are an untapped opportunity for getting work done more effectively. We’re taking this further by making sure this class of data will have enterprise-grade manageability and extensibility that integrates with existing applications and processes.
One of our earliest HoloLens partners, thyssenkrupp, is leveraging this work now to discover insights that will make it easier for its employees to complete some of their most important tasks.
Here is a look into the work our friends at thyssenkrupp are doing to leverage spatial analytics.

We will have a lot more to share in the coming weeks and months. Soon you will hear us share more on how we are going to help with training and development and collaboration on product design and prototyping. Our team loves doing work that really makes a difference for our customers!
And a reminder — if you want to be a part of the preview for Microsoft Remote Assist and Microsoft Layout, please visit us here.
As always, feel free to reach out to me on Twitter to share what you are doing with mixed reality.
Talk soon!

Introducing two new mixed reality business applications: Microsoft Remote Assist and Microsoft Layout
Tweet This

Mixed reality in manufacturing comes to life at Hannover Messe

I hope this month’s mixed reality blog post finds you well!
For this entry, I am going to focus on some of the amazing work we are seeing from customers and partners this week at the Hannover Messe (HM) show in Germany.
Last year, we highlighted some exciting customer stories, including how thyssenkrupp is transforming the delivery of home mobility solutions using Microsoft HoloLens.
This year, we are demonstrating how HoloLens and Windows Mixed Reality are helping companies reimagine processes, data, and people to accelerate business impact and create value for customers.
The manufacturing industry has been aggressively adopting new technologies that empower Firstline Workers to solve problems, achieve efficiencies, and better serve customers. Mixed reality is bringing these benefits to life, and today, manufacturers around the world are realizing cost savings and efficiency gains as a result.
Here is a view into some of the mixed reality solutions on display at the Microsoft booth this week at Hannover Messe
BAE Systems and PTC
BAE Systems makes the electric propulsion Systems for HybriDrive® buses. BAE is working with PTC, a Microsoft Mixed Reality Partner, utilizing their ThingWorx Studio to create mixed reality solutions that dramatically improve the efficiency of Firstline Workers. BAE and PTC used ThingWorx Studio to easily create a guided step-by-step training solution for HoloLens to teach workers how to assemble a green energy bus battery. Using these tools, BAE can now create these guides for firstline workers in just a few hours at a tenth of the cost, training new people 30-40% more efficiently.
Here is a look at how BAE Systems and PTC are using HoloLens:

Honeywell Maintenance Training Simulator
In February, our friends at Honeywell introduced the Honeywell Maintenance Training Simulator. This work empowers system maintenance personnel to be prepared for a variety of critical situations and to address them faster with reduced risk. It enables their employees to test and enhance their knowledge in a safe and simulated immersive environment prior to managing critical infrastructure.
Mercedes-Benz Global Training
Mercedes-Benz Microsoft HoloLens brake assembly
Mercedes-Benz Global Training is showing off how HoloLens can be used to improve the efficiency of brake assembly and engine manufacturing. In addition, they are demonstrating how customers can load a roof box with their virtual bags or sports items, such as skis and equipment in a real roof box mounted on an E-Class sedan.
Autodesk Project Sugarhill
Autodesk is demonstrating Project Sugarhill, a technical preview of immersive conceptual design tools that empower artists and designers to start their creative process directly in 3D. Using simple curve and surfacing tools, users can explore form and shape while being fully immersed alongside their design. 3D sketches and models can be easily exported to other screen-based applications such as Alias or Maya for final realization.
Deutsche Bahn
Deutsche Bahn railway training in mixed reality
Deutsche Bahn has worked with the Microsoft Mixed Reality Partner Program to build an app that will enable the German railway company to more efficiently and safely train engineers on railway parts and maintenance. Really cool to see them bring this to life on the show floor!
Microsoft Visio
This week, we’re delighted to demonstrate Microsoft Visio updates designed to address intelligent manufacturing scenarios, including facility layout. As part of our work at HM, we will be showcasing Intelligent Manufacturing scenarios supported by Microsoft customer Dürr AG and incorporating Microsoft Visio Partners FaciWare GmbH (facility management solutions) and X-Visual Technologies GmbH (PnID solutions).
It’s been exciting to watch mixed reality solutions for the manufacturing industry transition from proof-of-concept to production deployments that deliver tangible benefits to manufacturers. We remain inspired by the ways our customers and partners are using mixed reality and HoloLens to do things that were previously impractical or impossible.
I can’t wait to share more in the weeks and months ahead!
As always, feel free to reach out to me on Twitter @lorrainebardeen to share what you are doing with mixed reality.
Talk soon!

Mixed reality in manufacturing comes to life at Hannover Messe
Tweet This

Computer Aided Design (CAD) enters the era of mixed reality

I’m excited to share this month’s blog post with you all. In addition to highlighting some of the cool work we are seeing from the mixed reality community, I also have some important Simplygon news to share.
In the time since we acquired Simplygon they have been working quickly to bring the value of the Simplygon service to mixed reality.
Let’s dig in!
Bringing Computer Aided Design (CAD) into the era of mixed reality
Throughout history, media portability has been critical to the adoption of new computing experiences. For example, prior to the mid-1990s, compact disc owners were unable to move their music from physical to digital media. As audio encoding technology became ubiquitous, music and other digital files became much easier to move between devices. This portability enabled the journey from physical media to mobile mp3 players, to digital music stores, and finally to today’s world where access to nearly every song lies just a few clicks away.
We think about examples like this as we make investments to support the extensibility of 3D content across screens and into mixed reality. 3D content of all types – static, animated, simple, and complex – is key to transformative mixed reality experiences. This is why we’ve invested to support GL Transmission Format (glTF) (known by many as “JPEG of 3D”) as an industry standard 3D file format and to bring Simplygon’s 3D asset optimization and decimation services into the Microsoft family.
Today, I am excited to announce that Simplygon has added compatibility for JT and STEP file formats, which are pivotal to the productivity of CAD professionals. The addition of two of the most common CAD file formats brings automation of 3D optimization to customers across many industries. As the enterprise increasingly embraces the era of digital transformation, CAD support for Simplygon empowers more industries to painlessly and efficiently bring their existing assets into mixed reality.
For 3D content, this announcement is another early step along that same transformative path. CAD users are now able to create on their PC in JT or STEP formats, transcode and optimize via Simplygon, and distribute their work to their preferred device endpoint. We will continue taking steps along the journey to 3D portability and making the proliferation of 3D content as simple as possible.
JT and STEP are two of the most common CAD file types and are both ISO standards. These file types are heavily used in manufacturing industries and are vital to solving 3D problems. JT files, used heavily in product lifecycle management software, is primarily used to analyze and augment the geometry of complex models. STEP, or Standard for the Exchange of Product model data, is primarily used to share 3D models between users with different CAD software packages.
How to get started
Simplygon Cloud is available in Azure Marketplace today. To start, visit our Azure Marketplace home to learn how to deploy the Simplygon Cloud virtual machine and start optimizing your 3D assets. Please also visit our documentation for examples and more information on how to integrate this into your workflow today.
Continued adoption of mixed reality
During the month of March, we saw some fun, unique, and inspiring work from the global mixed reality community. Here were some of my favorites!
Toms River Municipal Utilities Authority (TRMUA) uses mixed reality to view simulated GIS features
This is so cool! Len Bundra, the IT/GIS director for the Toms River Municipal Utilities Authority (TRMUA) in Toms River, New Jersey and Alec Pestov, a developer from Toronto, Canada and Founder and CEO of Meemim worked together to create a mixed reality experience that would allow utilities (and eventually other organizations) to collaborate with shared content and data. When we talk about mixed reality and HoloLens giving people super-powers that change the way we work, this is exactly the type of innovation we are talking about!

Renault Sport Formula One team brings their creations to life with Microsoft HoloLens
I loved reading about the work that the Renault Sport Formula One Team is doing with HoloLens to make it easier for race fans to experience their one of a kind (literally!) creations. As Mark Everest, IS Development Manager at Renault Sport Formula One Team notes, “We build two cars a year, not thousands of cars. The innovation rate involved in design, simulation, testing and manufacturing is much faster than consumer car companies.”
Before each new season begins, an entirely new car is born. The Renault Sport Formula One Team has partnered with Microsoft to create a mixed reality experience that brings the car to life at events around the world – without the car (since there is only one) physically having to be there. And while the team is exploring how the HoloLens could impact engineering in the future, this experience will immerse viewers in the current evolution of design from earlier cars — like the RE40 (1983), R26 (2006), and R31 (2011) — to the most recent models, paired with facts about the design process and performance.

Reinventing Brian Eno and Peter Chilvers’ ‘Bloom’ for mixed rality
Musician, producer, visual artist, and thinker Brian Eno, together with longtime collaborator Peter Chilvers, launched their first mixed reality project with Bloom: Open Space, a generative audio-visual installation brought to life with HoloLens. Based on their award-winning app, Bloom, the innovative mixed reality installation blurs the lines between the physical and virtual, exploring uncharted territory in the realms of both applications and generative art.

Murmur: Arctic Realities at Mystic Seaport
I have written before about how inspired I am when people use mixed reality to tell stories and paint an otherwise blank canvas. This month I wanted to call attention to an exhibit that is coming to life at Mystic Seaport in New Haven, CT.
Murmur: Arctic Realities explores Arctic landforms called pingos – ice hills rising from permafrost soil, covered by a thin layer of earth. These hills grow over centuries and then collapse, pockmarking the landscape in the Circumpolar North.
Seattle artist John Grade and new media artist Reilly Donovan have partnered to present an immersive technological component for Murmur using HoloLens. Through HoloLens, Grade and Donovan have created a holographic patchwork of tundra, complete with water, living organisms, and the sound of Arctic birds. As visitors move about the gallery they will notice that aspects of the experience respond to their presence.

I look forward to sharing our update for next month where we will dive into the work we are doing along with our customers and partners at Hannover Messe, one of my many favorite industry events.
Talk soon!
(P.S. – feel free to follow me on Twitter @lorrainebardeen where I regularly share updates and answer questions from the mixed reality community)

Computer Aided Design (CAD) enters the era of mixed reality
Tweet This

How mixed reality is changing the game for healthcare, from performing live surgeries to delivering ultrasounds in 3D

Last year around this time I had the chance to write a post about one of my favorite mixed reality uses cases – healthcare.
Since that time, we have continued to see remarkable momentum for mixed reality across numerous facets of the industry. This week at HIMSS our team had the privilege to once again be on the ground to see and experience how customers and partners are embracing mixed reality to transform the way they innovate, support, and help others.
Coming out of the show I wanted to take a moment to spotlight some of the innovation work we are seeing from organizations who are working to forge the future of healthcare.
CAE Healthcare
Joining us onsite at HIMSS this year was CAE Healthcare. CAE is an industry leader in the development of simulation-based technologies, curriculum, and resources that improve clinical performance. I talked about some of their earliest work with Microsoft HoloLens last year – well, they certainly haven’t slowed down!
CAE VimedixAR is the first ultrasound simulator to integrate HoloLens. Freed from the limits of a two-dimensional environment inside a monitor, healthcare professionals can display, enlarge, turn, and rotate realistic-looking anatomical parts, or command them to return into the manikin body. Users and those learning with this technology are able to witness (in real time) the ultrasound beam as it cuts through human anatomy.

With CAE LucinaAR, clinical learners can view 3D holograms of a fetus as it descends the birth canal and gain an unprecedented view of anatomy as they learn to manage a shoulder dystocia delivery. CAE Lucina is a wireless childbirth simulator with validated, integrated maternal-fetal physiology for training on all the stages of delivery and the rare emergency scenario. With five pre-configured Simulated Clinical Experiences (SCEs), and new mixed reality modules coming soon, Lucina allows instructors at universities and hospitals to easily integrate simulation and augmented reality into their hands-on educational programs.

With their Abiomed Impella procedural training solution, CAE Healthcare is integrating HoloLens into its simulation-based training platform that allows physicians to practice ultrasound-guided placement of the world’s smallest heart pump. Overlaying the ultrasound manikin, probe and instruments, the CAE HoloLens framework will depict physiological responses and potential complications to help clinicians quickly master and deploy the device without risk to patients.
Adding to its physics-driven simulator for the Medtronic Micra Transcatheter Pacing System (TPS), the world’s smallest pacemaker, CAE Healthcare delivered a mixed reality training solution for up to 11 simultaneous learners on its Microsoft HoloLens platform. Learners can practice while viewing interactive holograms of a 3D beating heart into which the Micra delivery system is visible and connected in real-time.
Pearson delivers mixed reality learning tools for nursing professionals
At Bett this year we were thrilled to announce some of the great curricula our friends at Pearson were delivering to train nurses using mixed reality and the benefit of holographic patients. Today, I am happy to share more details about one of their extraordinary pieces of the curriculum – HoloPatient.
HoloPatient is a mixed-reality learning tool for nursing, allied health and medical schools that will deliver simulated patient experiences to more students, at a more manageable cost, and with a greater degree of consistency than ever before. This cutting-edge 3D immersive solution redefines how learners interact with their worlds and each other—bringing learning to life with startling clarity and breathing new possibility into teaching. Designed and conceived in collaboration with expert healthcare educators and simulation facilitators from around the world, HoloPatient uses Microsoft’s Mixed Reality Capture technology allowing tutors and students to engage in a range of learning and assessment activities previously only possible when using actors or by encountering patients in real life.

HoloPatient drives consistency across the delivery of learning content and assessment while also allowing nursing and medical schools to augment their existing delivery of world-class education and training with one of the most promising and engaging teaching and learning use-cases for mixed reality currently available.
The HoloPatient app will be released globally in April 2018, along with a virtual reality version later in the year. By the end of June 2018, Pearson will have released four other learning tools including Anatomy, Chemistry (building molecules in 3D), Math (Area and Volume of 3D objects) and History.
You can find out more at
SphereGen is utilizing mixed reality to create innovative solutions designed to increase efficiency and revolutionize the learning process. SphereGen collaborated with St. George’s University to develop the Learning Heart. The Learning Heart is a HoloLens application that assists students in understanding the elements of the Heart. As a study aid and teaching tool, this application can be used individually or collaboratively by multiple HoloLens users. The holograms are fully three-dimensional and can be traversed around and viewed from any position and angle.

DICOM Director is a full product suite designed to enable communication and collaboration between different radiologists, doctors, and medical practices. The backend of this solution enables radiologists to view, transfer and share scans. This solution utilizes HoloLens as a viewer for radiological scans, such as MRIs, cat scans, and several others. Radiologists and doctors can remotely view their scans through the HoloLens and/or a Windows Mixed Reality headset. The viewing functionality enables the user to see the series of scans, an enlarged view of the selected scan (which they can scroll through), and the 3D model reconstruction from the scans.

Live surgery with HoloLens
One of the most mind-blowing moments in mixed reality occurred just a few months ago. On that day AP-HP, Paris French hospitals group, conducted a live surgery on YouTube, assisted by TeraRecon medical holographic platform leveraging Azure GPU and HoloLens. The surgeon was assisted by TeraRecon and their partners to develop the complete end-to-end solution from the patient surgery planning up to the real-time streaming of the real patent image data (pre-operative 3D CT scan) with its full volume rendering in the highest quality ever seen including segmentation of the CT scan. This live surgery was conduct using sharing experience to share the same patient data across multiple HoloLens with the same holographic space (operating room) as well as sharing the HoloLens view with the remote surgeon.
With a mission to empower doctors, Visual 3D Medical Science and Technology Development CO. LLC) is dedicated to the use of artificial intelligence, cloud computing, and mixed reality to enhance the capability of doctors. By way of research and development, Visual3D’s technology can promote doctor’s ability of perception and appliance of medical information; enhance the remote cooperative abilities of doctors in different places to build a scientific and technological platform for intelligent medical therapy.
Visual3D has already performed over 200 hip, knee, and spinal surgeries in China using HoloLens. In addition to using HoloLens in the operating theater, Visual3D is using HoloLens to assist doctors pre-surgery as both a tool to brief patients on their upcoming surgical procedures, as well as allowing doctors to pre-visualize and rehearse the procedure before entering the operating room.
Next Surgeries
Digital Pages, a leading Brazilian company in the lifecycle management of digital contents, has developed an interactive application for surgery planning using Microsoft HoloLens. The application offers students, surgeons and healthcare professionals a new perspective on the modeling and manipulation of physical and virtual 3D objects. Next Surgeries introduces the use of holographic projections in surgical planning, reducing the total time of surgeries, the exposure of the patient and staff to radiation and risk of infections, blood loss, etc. The application is available on the digital teaching platform RDP Learning integrated to HoloLens, bringing innovation and more security to the health sector.
I look forward to sharing more with you all soon as we continue to shine a light on many of the great mixed reality innovations being delivered by customers, partners, and developers of mixed reality!

#ifdef WINDOWS – One Developer’s Guide to the Surface Hub

Building experiences for the Surface Hub requires developers to tailor their apps for the big screen and multiple users interacting with different components at the same time. Gian Paolo Santopaolo came to Redmond all the way from Switzerland to share best practices for developing engaging and natural user experiences for the Surface Hub. He has spent the last 6 years as a Windows MVP, designing and developing natural user experiences for various devices including the Microsoft Surface Hub and Microsoft HoloLens. During this time, Gian Paolo has developed many open source components and helpers to enable developers to build better and more personal UWP apps for every device and scenario.
Watch the full video above and feel free to reach out on Twitter or in the comments below for questions or comments.
Happy coding!

Mixed Reality at Microsoft – February update

It’s hard to believe that 2018 is almost two months old! Speaking on behalf of everyone on the mixed reality team at Microsoft, we are excited about the year to come. We have a lot of fun things planned.
Because I love mixed reality so much, I thought February 14th (Valentine’s Day for those who celebrate) would be the perfect day to kick off a new regular update from us. Each month we will share some news on what we are doing, and we will highlight some of the great work coming to market from our customers and partners.
Let’s get going with what we have to share today!
Making it easier to get your hands on HoloLens
We have heard loud and clear that people are looking for additional ways to get HoloLens. To support that demand we have two important program updates to share.
HoloLens expands to more markets
First, I’m happy to announce we will soon be making HoloLens available in Singapore and the United Arab Emirates. To all the developers, creators, partners, and customers we will now work with – welcome! With these new additions, HoloLens is now available in 41 markets around the world.
HoloLens now available for rent
Second, many of our customers have expressed a desire to rent HoloLens, so that companies can evaluate before purchasing or increase their inventory temporarily to support tradeshows and events.
Today, we are excited to help remove this barrier and announce that customers in North America can now rent a HoloLens from our partners at ABCOMRENTS. We are also working to bring this program to additional markets in the months ahead. We look forward to sharing those details soon.
For those in the USA and Canada who are interested in renting HoloLens devices, please visit this link.
Continued adoption of mixed reality
The best part of my job is seeing what people around the world are doing with mixed reality. The innovation and development we see on the platform inspires us to create the software and tools needed to bring the potential of mixed reality to life.
Over the first six weeks of 2018, we have seen some really great work from our partners and customers. Here are a few of my favorites.
Trimble expands their mixed reality product portfolio

Last month, Trimble announced Trimble Connect for HoloLens and a new hard hat solution for HoloLens that improves the utility of mixed reality for practical field applications. Trimble has paid close attention to how to support HoloLens as a high-value tool for firstline workers and are continuing to increase their impact on the market.
Trimble Connect for HoloLens is a mixed-reality solution that improves coordination by combining models from multiple stakeholders such as structural, mechanical and electrical trade partners. The solution provides for precise alignment of holographic data on a 1:1 scale on the job site, to review models in the context of the physical environment. Trimble Connect for HoloLens is available now through the Microsoft Store.
Trimble’s Hard Hat Solution for Microsoft HoloLens extends the benefits of HoloLens mixed reality into areas where increased safety requirements are mandated, such as construction sites, offshore facilities, and mining projects. The solution, which is ANSI-approved, integrates the HoloLens holographic computer with an industry-standard hard hat. Trimble’s Hard Hat Solution for HoloLens is expected to be available in the first quarter of 2018.
HP unveils Windows Mixed Reality Headset – Professional Edition
Last week, we saw HP announce the HP Windows Mixed Reality Headset – Professional Edition. What we love about this headset is the work HP is doing to enhance the way work is done within a set of industries we also care deeply about – Engineering Product Dev and design reviews, AEC (Architecture, Engineering & Construction) reviews, and MRO (Maintenance, Repair and Overhaul) training use environments.

Just like the consumer edition, this headset delivers 1440×1440 resolution per eye and up to a 90Hz refresh rate. For businesses, this headset is the perfect combination of comfort and convenience as it comes with easy to clean, replaceable face cushions. It also uses a double-padded headband, easy adjustment knob and front-hinged display for a superb experience for one or more users.
Honeywell introduces mixed reality simulator to train industrial workforce and help close skills gap

Earlier this week Honeywell Process Solutions announced a cloud-based simulation tool named Honeywell Connected Plant Skills Insight Immersive Competency. This new offering uses mixed reality to train plant personnel on critical work activities. With as much as 50 percent of industrial plant personnel due to retire within the next five years, Immersive Competency is designed to bring new industrial workers up–to–speed quickly by enhancing training and delivering it in new and contemporary ways. By implementing experiences that take advantage of both HoloLens and Windows Mixed Reality immersive headsets, Honeywell has been able to deploy a solution that directly links industrial staff competency to plant performance by measuring the training’s effectiveness based on real outcomes.
We look forward to sharing more next month. If you have any questions or needs, feel free to reach out to me on Twitter – @lorrainebardeen.
And – if you celebrate – enjoy Valentine’s Day! Here’s a little something for you.


Mixed Reality at Microsoft – February update
Tweet This

Microsoft Education unveils new Windows 10 devices starting at $189, Office 365 tools for personalized learning, and curricula to ignite a passion for STEM

Our ongoing commitment to empower every student to create the world of tomorrow continues today with several new announcements for education. Whether igniting interest in STEM, learning in 3D, helping students of all abilities improve their reading, writing, and comprehension, or saving teachers time and helping them collaborate, we know technology has the power to unlock limitless learning.
I love hearing about the inspiring results we see from schools using Microsoft Education products, and we’ve been pleased to see more schools choosing Windows this year.
This week at Bett, we’ll show new Windows 10 and Windows 10 S devices from Lenovo and JP, starting at just $189, providing more options for schools who don’t want to compromise on Chromebooks. We’ll add new capabilities to our free Office 365 for Education software, enabling any student to write a paper using only their voice and making it easier to access Teams via mobile devices. And we’re making STEM learning fun with a new Chemistry update to Minecraft: Education Edition and new mixed reality and video curricula from partners like BBC Worldwide Learning, LEGO®* Education, PBS, NASA, and Pearson.
New Windows 10 devices starting at $189 and free professional development
Affordability is the top priority for many schools. Shrinking budgets can lead schools to choose devices with a stripped-down experience and a limited lifespan, unfortunately costing more over time and offering less to students. Our new Windows 10 devices won’t force schools to compromise on features, giving schools cutting-edge tools like touch, inking, and 3D as well as free accessibility technology like Learning Tools, which improve reading comprehension by 10 percentile points for students of all abilities. This week, building on the success of our expansive education device portfolio, we’ll be showing new 2-in-1s under $300 and notebooks under $200. Our new devices include:
Two new Lenovo devices to give schools great performance at a great price.

The Lenovo 100e, a brand-new Intel Celeron Apollo Lake powered PC, starting at just $189 USD**.

The Lenovo 300e, an affordable 2-in-1 convertible PC with pen support, starting at $279 USD.
Two new devices from JP, one of our largest partners in emerging markets.

The Classmate Leap T303 laptop with Windows Hello, starting at $199 USD.

The Trigono V401 2-in-1 with pen and touch, starting at $299 USD.
These new devices join the HP ProBook x360 11 EE, which continues to be one of our best-selling 2-in-1 devices in education at just $299 USD and the recently released HP Stream 11 Pro G4 EE PC starting at $225 USD. They are all spill resistant and ruggedized to avoid accidental breakage, have long battery life to avoid charging wires all over the classroom and have faster connectivity.
With Intune for Education, it’s easy to set up and manage Windows devices. With Microsoft 365 Education schools can get our complete software solution with free Office 365 Education and starting next month, with purchase of certain Microsoft 365 Education licenses, they can also get free Professional Development to help train teachers on how to get the most from their new technology.***
Personalized learning drives better results
Office 365 is built to support every type of learner, with assistive technology natively built-in and tools like Word, OneNote, and Teams that give every child a voice in the classroom. We are proud to offer teachers in 146 countries Office 365 Education for free, and there are a number of new updates coming to Office in the next month, including:
Starting in February, we will introduce dictation in Office 365 to help students write more easily by using their voice.
Microsoft Learning Tools improves reading, writing, and comprehension for all students, especially the one in five students who have a learning difference. Starting in February, we will introduce dictation in Office 365 to help students write more easily by using their voice. Immersive Reader functionality will also expand Word for Mac, iPhone, Outlook Desktop, OneNote iPad, and OneNote Mac with support for many new languages.
OneNote Class Notebook, which has added more than 15 million new student notebooks since the beginning of this school year, will now include assignment and grade integration with the most widely used School Information Systems in the UK (SIMS Capita).
To further simplify classroom workflows, we are delivering on the number one request from teachers for Class Notebooks, enabling them to lock pages as read-only after giving feedback. In OneNote, we are also enabling Desmos interactive math calculators, a set of popular applications for STEM teachers.
Teams, a digital hub for the classroom, is now accessible on iOS and Android so teachers and students can now keep track of their assignments and classroom conversations on their phones or tablets. To enable language learners to engage in classroom conversations, our newest updates will also make it easy to translate any conversation or chat into another language.
PowerPoint will now allow teachers to record their lessons including slides, interactive ink, video, and narrations and publish to their Stream channel in Teams classrooms. This way, students can view from anywhere in advance of class, so class time can be used for conversation. Stream will also add automatic captioning to the videos to make them accessible to all learners.
Our recently announced partnership with PowerSchool – the leading global education technology platform for K-12 – will help create classroom efficiencies, enable collaboration and provide insights that allow teachers to spend more time teaching and less time focused on administrative tasks.
Sparking creativity with STEM
Igniting a passion for STEM in today’s students will help prepare them for the future. We’re committed to giving teachers new ways to teach these subjects with fun curricula that sparks students’ innate creativity, a skill that is predicted to be one of the top three most desired skills by employers in 2020.
One way we do this is through MakeCode, which today announced a new Cue Education app, available first on Windows, new curricula for Computer Science with MakeCode for Minecraft, and a new MakeCode for micro: bit Windows 10 app.
Chemistry comes to life through Minecraft
Nothing shows the power of creativity in STEM education more than Minecraft: Education Edition, which is already being used in classrooms in more than 115 countries. This spring, a new and free Chemistry Update will enable teachers to use the familiar world of Minecraft and game-based learning to engage students of all ages in chemistry. Through hands-on experimentation, students can learn everything from building compounds to more difficult topics like stable isotopes.
Mixed reality for immersive learning
Our research shows the most effective learning involves multiple regions of the brain. In the era of mixed reality – which includes augmented and virtual reality, we are working with our partners to deliver products and experiences that will help teachers push the boundaries of their curricula.
Over the last year, we have seen remarkable instances of teachers and institutions using Microsoft HoloLens to bring 3D experiences to life in the classroom. With the release of Windows Mixed Reality immersive VR headsets that start at $299, we provide a more affordable entry point for mixed reality in the classroom.
To support that effort, today we are announcing three important program updates:

Pearson – the world’s largest education company – will begin rolling out in March curriculum that will work on both HoloLens and Windows Mixed Reality immersive VR headsets. These six new applications will deliver seamless experiences across devices and further illustrate the value of immersive educational experiences.
We are expanding our mixed media reality curriculum offerings through a new partnership with WGBH’s Bringing the Universe to America’s Classrooms project****, for distribution nationally on PBS LearningMedia. This effort brings cutting-edge Earth and Space Science content into classrooms through digital learning resources that increase student engagement with science phenomena and practices.
To keep up with growing demand for HoloLens in the classroom we are committed to providing affordable solutions. Starting on January 22, we are making available a limited-time academic pricing offer for HoloLens. To take advantage of the limited-time academic pricing offer, please visit,
New partnerships in STEM learning
Lesson plans that encourage hands-on and visual learning drive deeper engagement from students, and we’re excited to partner with some of the world’s most respected brands to bring teachers exciting new lessons in time for the next school year.
Starting this March, we’ll be partnering with BBC Worldwide Learning to bring the stunning BBC Earth natural history film Oceans: Our Blue Planet to classrooms and museums around the world, starting with the release of the film in museums this March. As part of the multi-faceted partnership, we are also exploring ways to expand this content across the Microsoft Education portfolio.
Starting this March, we’ll be partnering with BBC Worldwide Learning to bring the stunning BBC Earth natural history film Oceans: Our Blue Planet to classrooms and museums around the world, starting with the release of the film in museums this March.
And together with LEGO® Education, we’ll be offering a new free online Hacking STEM lesson plan that has students use the Pythagorean Theorem to explore and measure topography in 2D/3D space. Acting as environmental surveyors and engineers, students build tools with LEGO® bricks from the Simple & Powered Machines set or cardboard to create and visualize an initial transportation plan for the development of an island National Park in Excel. They then bring their National Park to life by adding topographic elements.
With all of these new experiences available in time for the upcoming school year, there’s never been a better time for schools to try Microsoft Education. We will continue to work hard to deliver innovative technology for educators around the world to unlock limitless learning for students of all abilities.
*LEGO, the LEGO logo are trademarks and/or copyrights of the LEGO Group. ©2018 The LEGO Group. All rights reserved. **$219 global pricing ***If at least 30 qualified licenses are purchased between February 1, 2018, and December 31, 2018, you can schedule free in-person or virtual Professional Development led by certified Microsoft training partners. Learn more on **** “Bringing the Universe to America’s Classrooms,” is a cooperative agreement awarded to WGBH from the National Aeronautics and Space Administration (NASA) under cooperative agreement award No. NNX16AD71A.

#ifdef WINDOWS – Getting Started with Mixed Reality

I sat down with Vlad Kolesnikov from the PAX Mixed Reality team in Windows to talk about what mixed reality is and how developers can get started building immersive experiences. Vlad covered what hardware and tools are needed, building world-scale experiences, and using the Mixed Reality Toolkit to accelerate development for Microsoft HoloLens and Windows Mixed Reality Headsets.
Check out the video above for the full overview and feel free to reach out on Twitter or in the comments below.
Happy coding!

Watching in awe as artists bring their visions to life in mixed reality

I’ll never forget the first time I experienced mixed reality.
It was a pre-production version of Microsoft HoloLens, and I tried a demo that had fish swimming around the room, seaweed on the floor and bubbles floating around me.  Eventually, the demo team had to gently suggest I leave, because I was mesmerized.  The fish were bouncing off the walls. The bubbles were popping on the ceiling.  I could see everything in the room and the people in it, and I could see more – things that felt just as real.  I couldn’t get enough.
Part of me is still in that moment because it’s what has motivated me to work as hard as I can with teams across Microsoft to build the devices, platforms, experiences and services that will enable infinite mixed reality experiences.
Everyone working in mixed reality today is a pioneer as they push forward, defining boundaries of technology, design and user experience while reinventing industries, education and communication.
Today, I want to talk about one group of pioneers who are using mixed reality to change the way we see the world: artists.

Since the early days of HoloLens, we have had the chance to see what happens when artists use their unique ability to see the potential of a new medium or technology. I have always been passionate about paying attention to what artists are doing with new technology platforms.  Artists, whether visual artists, media, sculpture, dance, theater or others, have both courage and the ability to see things differently.
It has been amazing to see what has been created.
Cisneros Fontanals Art Foundation Exhibit at Art Basel
Last week we kicked off a partnership with Cisneros Fontanals Art Foundation (CIFO) at Art Basel in Miami. Their exhibit, “Sandu Darie: An Immersive Experience,” aims to make important Cuban works of art accessible beyond the constraints of their physical location. The experience makes available public works of art that are permanently located in Cuba and brings them to life using Mixed Reality Viewer, Windows Mixed Reality headsets and HoloLens. For the first time outside of Cuba, visitors could view large-scale works of art in their real world through these two immersive experiences.
A photographer captures images of Cuban artist Sandu Darie’s El Día y La Noche (1982) in Havana that were used to digitally recreate the sculptural mural as a hologram
Case Western Reserve University mixed reality dance shared experience
Case Western Reserve University has truly been a pioneer in mixed reality. From the building of anatomy class experiences, to expanding to physics classes, and more recently exploring the arts with the British Museum and others, the team at Case has been leading from the edge.
I recently had the opportunity to have another once-in-a-lifetime experience: the first-ever, large-scale shared mixed reality dance performance. Eighty people all wore HoloLens to see dancers and holograms in the same performance, for the first time.  The team had no idea how to pull this off when it was conceived by the Creative Director, XX, but that’s why we’re in this space: to bring things that seem impossible into focus, and then to just figure out how to do them.
Take a look at their blog post and video to learn more about how the vision ultimately came to life.
Cornish College of the Arts
I spent two years in early HoloLens development saying, “Imagine when you go to the first holographic art gallery opening.  There will be wine and cheese, maybe some light music, but no art.  You walk in and see empty walls, empty stands. But then you put on HoloLens and the gallery is full of rich art.”
I would marvel over this with other people.
Well, this has happened, and much earlier than I expected.
Cornish College of the Arts pioneered the first exhibit in 2016, called “Through the HoloLens,” which allows students to bring to life their artistic expression in new ways through painting, dance, performance and other forms. I was fortunate to experience this firsthand with the students of Cornish, a moment I will never forget.
Gallery goers check out students’ holographic works at the Cornish College of the Arts 2016 BFA show
Artsy at the Armory
Earlier this year we partnered with online art platform Artsy and Amsterdam-based Studio Drift to create a mixed reality art installation at The Armory Show in New York.  The project was called “Concrete Storm” and was our first collaboration in a commercial art context.  It blended physical structures with holograms, which changed perspective depending on where you stood within the exhibit, allowing visitors to experience art in an entirely new way. It was fascinating to watch the typically jaded New York art crowd’s expressions the first time they put the device on; lighting up as they walked through the concrete pillars and reaching out to touch something that wasn’t really there. Proof that mixed reality can make art more accessible and interactive when it comes alive, up close and personal.
Studio Drift founders Lonneke Gordijn and Ralph Nauta view their original HoloLens work, Concrete Drift (2017), at The Armory Show in New York
I’d like to thank all working artists for your commitment to your craft and work.  And for those of you who are deeply engaged with technology, thank you again.  Your work is so very important. As we get ready for the year ahead I am eager to see what comes next.