Category Archives: Microsoft HoloLens

Auto Added by WPeMatico

#ifdef WINDOWS – One Developer’s Guide to the Surface Hub

Building experiences for the Surface Hub requires developers to tailor their apps for the big screen and multiple users interacting with different components at the same time. Gian Paolo Santopaolo came to Redmond all the way from Switzerland to share best practices for developing engaging and natural user experiences for the Surface Hub. He has spent the last 6 years as a Windows MVP, designing and developing natural user experiences for various devices including the Microsoft Surface Hub and Microsoft HoloLens. During this time, Gian Paolo has developed many open source components and helpers to enable developers to build better and more personal UWP apps for every device and scenario.
Watch the full video above and feel free to reach out on Twitter or in the comments below for questions or comments.
Happy coding!

Mixed Reality at Microsoft – February update

It’s hard to believe that 2018 is almost two months old! Speaking on behalf of everyone on the mixed reality team at Microsoft, we are excited about the year to come. We have a lot of fun things planned.
Because I love mixed reality so much, I thought February 14th (Valentine’s Day for those who celebrate) would be the perfect day to kick off a new regular update from us. Each month we will share some news on what we are doing, and we will highlight some of the great work coming to market from our customers and partners.
Let’s get going with what we have to share today!
Making it easier to get your hands on HoloLens
We have heard loud and clear that people are looking for additional ways to get HoloLens. To support that demand we have two important program updates to share.
HoloLens expands to more markets
First, I’m happy to announce we will soon be making HoloLens available in Singapore and the United Arab Emirates. To all the developers, creators, partners, and customers we will now work with – welcome! With these new additions, HoloLens is now available in 41 markets around the world.
HoloLens now available for rent
Second, many of our customers have expressed a desire to rent HoloLens, so that companies can evaluate before purchasing or increase their inventory temporarily to support tradeshows and events.
Today, we are excited to help remove this barrier and announce that customers in North America can now rent a HoloLens from our partners at ABCOMRENTS. We are also working to bring this program to additional markets in the months ahead. We look forward to sharing those details soon.
For those in the USA and Canada who are interested in renting HoloLens devices, please visit this link.
Continued adoption of mixed reality
The best part of my job is seeing what people around the world are doing with mixed reality. The innovation and development we see on the platform inspires us to create the software and tools needed to bring the potential of mixed reality to life.
Over the first six weeks of 2018, we have seen some really great work from our partners and customers. Here are a few of my favorites.
Trimble expands their mixed reality product portfolio

Last month, Trimble announced Trimble Connect for HoloLens and a new hard hat solution for HoloLens that improves the utility of mixed reality for practical field applications. Trimble has paid close attention to how to support HoloLens as a high-value tool for firstline workers and are continuing to increase their impact on the market.
Trimble Connect for HoloLens is a mixed-reality solution that improves coordination by combining models from multiple stakeholders such as structural, mechanical and electrical trade partners. The solution provides for precise alignment of holographic data on a 1:1 scale on the job site, to review models in the context of the physical environment. Trimble Connect for HoloLens is available now through the Microsoft Store.
Trimble’s Hard Hat Solution for Microsoft HoloLens extends the benefits of HoloLens mixed reality into areas where increased safety requirements are mandated, such as construction sites, offshore facilities, and mining projects. The solution, which is ANSI-approved, integrates the HoloLens holographic computer with an industry-standard hard hat. Trimble’s Hard Hat Solution for HoloLens is expected to be available in the first quarter of 2018.
HP unveils Windows Mixed Reality Headset – Professional Edition
Last week, we saw HP announce the HP Windows Mixed Reality Headset – Professional Edition. What we love about this headset is the work HP is doing to enhance the way work is done within a set of industries we also care deeply about – Engineering Product Dev and design reviews, AEC (Architecture, Engineering & Construction) reviews, and MRO (Maintenance, Repair and Overhaul) training use environments.

Just like the consumer edition, this headset delivers 1440×1440 resolution per eye and up to a 90Hz refresh rate. For businesses, this headset is the perfect combination of comfort and convenience as it comes with easy to clean, replaceable face cushions. It also uses a double-padded headband, easy adjustment knob and front-hinged display for a superb experience for one or more users.
Honeywell introduces mixed reality simulator to train industrial workforce and help close skills gap

Earlier this week Honeywell Process Solutions announced a cloud-based simulation tool named Honeywell Connected Plant Skills Insight Immersive Competency. This new offering uses mixed reality to train plant personnel on critical work activities. With as much as 50 percent of industrial plant personnel due to retire within the next five years, Immersive Competency is designed to bring new industrial workers up–to–speed quickly by enhancing training and delivering it in new and contemporary ways. By implementing experiences that take advantage of both HoloLens and Windows Mixed Reality immersive headsets, Honeywell has been able to deploy a solution that directly links industrial staff competency to plant performance by measuring the training’s effectiveness based on real outcomes.
We look forward to sharing more next month. If you have any questions or needs, feel free to reach out to me on Twitter – @lorrainebardeen.
And – if you celebrate – enjoy Valentine’s Day! Here’s a little something for you.

Lorraine

Mixed Reality at Microsoft – February update
Tweet This

Microsoft Education unveils new Windows 10 devices starting at $189, Office 365 tools for personalized learning, and curricula to ignite a passion for STEM

Our ongoing commitment to empower every student to create the world of tomorrow continues today with several new announcements for education. Whether igniting interest in STEM, learning in 3D, helping students of all abilities improve their reading, writing, and comprehension, or saving teachers time and helping them collaborate, we know technology has the power to unlock limitless learning.
I love hearing about the inspiring results we see from schools using Microsoft Education products, and we’ve been pleased to see more schools choosing Windows this year.
This week at Bett, we’ll show new Windows 10 and Windows 10 S devices from Lenovo and JP, starting at just $189, providing more options for schools who don’t want to compromise on Chromebooks. We’ll add new capabilities to our free Office 365 for Education software, enabling any student to write a paper using only their voice and making it easier to access Teams via mobile devices. And we’re making STEM learning fun with a new Chemistry update to Minecraft: Education Edition and new mixed reality and video curricula from partners like BBC Worldwide Learning, LEGO®* Education, PBS, NASA, and Pearson.
New Windows 10 devices starting at $189 and free professional development
Affordability is the top priority for many schools. Shrinking budgets can lead schools to choose devices with a stripped-down experience and a limited lifespan, unfortunately costing more over time and offering less to students. Our new Windows 10 devices won’t force schools to compromise on features, giving schools cutting-edge tools like touch, inking, and 3D as well as free accessibility technology like Learning Tools, which improve reading comprehension by 10 percentile points for students of all abilities. This week, building on the success of our expansive education device portfolio, we’ll be showing new 2-in-1s under $300 and notebooks under $200. Our new devices include:
Two new Lenovo devices to give schools great performance at a great price.

The Lenovo 100e, a brand-new Intel Celeron Apollo Lake powered PC, starting at just $189 USD**.

The Lenovo 300e, an affordable 2-in-1 convertible PC with pen support, starting at $279 USD.
Two new devices from JP, one of our largest partners in emerging markets.

The Classmate Leap T303 laptop with Windows Hello, starting at $199 USD.

The Trigono V401 2-in-1 with pen and touch, starting at $299 USD.
These new devices join the HP ProBook x360 11 EE, which continues to be one of our best-selling 2-in-1 devices in education at just $299 USD and the recently released HP Stream 11 Pro G4 EE PC starting at $225 USD. They are all spill resistant and ruggedized to avoid accidental breakage, have long battery life to avoid charging wires all over the classroom and have faster connectivity.
With Intune for Education, it’s easy to set up and manage Windows devices. With Microsoft 365 Education schools can get our complete software solution with free Office 365 Education and starting next month, with purchase of certain Microsoft 365 Education licenses, they can also get free Professional Development to help train teachers on how to get the most from their new technology.***
Personalized learning drives better results
Office 365 is built to support every type of learner, with assistive technology natively built-in and tools like Word, OneNote, and Teams that give every child a voice in the classroom. We are proud to offer teachers in 146 countries Office 365 Education for free, and there are a number of new updates coming to Office in the next month, including:
Starting in February, we will introduce dictation in Office 365 to help students write more easily by using their voice.
Microsoft Learning Tools improves reading, writing, and comprehension for all students, especially the one in five students who have a learning difference. Starting in February, we will introduce dictation in Office 365 to help students write more easily by using their voice. Immersive Reader functionality will also expand Word for Mac, iPhone, Outlook Desktop, OneNote iPad, and OneNote Mac with support for many new languages.
OneNote Class Notebook, which has added more than 15 million new student notebooks since the beginning of this school year, will now include assignment and grade integration with the most widely used School Information Systems in the UK (SIMS Capita).
To further simplify classroom workflows, we are delivering on the number one request from teachers for Class Notebooks, enabling them to lock pages as read-only after giving feedback. In OneNote, we are also enabling Desmos interactive math calculators, a set of popular applications for STEM teachers.
Teams, a digital hub for the classroom, is now accessible on iOS and Android so teachers and students can now keep track of their assignments and classroom conversations on their phones or tablets. To enable language learners to engage in classroom conversations, our newest updates will also make it easy to translate any conversation or chat into another language.
PowerPoint will now allow teachers to record their lessons including slides, interactive ink, video, and narrations and publish to their Stream channel in Teams classrooms. This way, students can view from anywhere in advance of class, so class time can be used for conversation. Stream will also add automatic captioning to the videos to make them accessible to all learners.
Our recently announced partnership with PowerSchool – the leading global education technology platform for K-12 – will help create classroom efficiencies, enable collaboration and provide insights that allow teachers to spend more time teaching and less time focused on administrative tasks.
Sparking creativity with STEM
Igniting a passion for STEM in today’s students will help prepare them for the future. We’re committed to giving teachers new ways to teach these subjects with fun curricula that sparks students’ innate creativity, a skill that is predicted to be one of the top three most desired skills by employers in 2020.
One way we do this is through MakeCode, which today announced a new Cue Education app, available first on Windows, new curricula for Computer Science with MakeCode for Minecraft, and a new MakeCode for micro: bit Windows 10 app.
Chemistry comes to life through Minecraft
Nothing shows the power of creativity in STEM education more than Minecraft: Education Edition, which is already being used in classrooms in more than 115 countries. This spring, a new and free Chemistry Update will enable teachers to use the familiar world of Minecraft and game-based learning to engage students of all ages in chemistry. Through hands-on experimentation, students can learn everything from building compounds to more difficult topics like stable isotopes.
Mixed reality for immersive learning
Our research shows the most effective learning involves multiple regions of the brain. In the era of mixed reality – which includes augmented and virtual reality, we are working with our partners to deliver products and experiences that will help teachers push the boundaries of their curricula.
Over the last year, we have seen remarkable instances of teachers and institutions using Microsoft HoloLens to bring 3D experiences to life in the classroom. With the release of Windows Mixed Reality immersive VR headsets that start at $299, we provide a more affordable entry point for mixed reality in the classroom.
To support that effort, today we are announcing three important program updates:

Pearson – the world’s largest education company – will begin rolling out in March curriculum that will work on both HoloLens and Windows Mixed Reality immersive VR headsets. These six new applications will deliver seamless experiences across devices and further illustrate the value of immersive educational experiences.
We are expanding our mixed media reality curriculum offerings through a new partnership with WGBH’s Bringing the Universe to America’s Classrooms project****, for distribution nationally on PBS LearningMedia. This effort brings cutting-edge Earth and Space Science content into classrooms through digital learning resources that increase student engagement with science phenomena and practices.
To keep up with growing demand for HoloLens in the classroom we are committed to providing affordable solutions. Starting on January 22, we are making available a limited-time academic pricing offer for HoloLens. To take advantage of the limited-time academic pricing offer, please visit, hololens.com/edupromo.
New partnerships in STEM learning
Lesson plans that encourage hands-on and visual learning drive deeper engagement from students, and we’re excited to partner with some of the world’s most respected brands to bring teachers exciting new lessons in time for the next school year.
Starting this March, we’ll be partnering with BBC Worldwide Learning to bring the stunning BBC Earth natural history film Oceans: Our Blue Planet to classrooms and museums around the world, starting with the release of the film in museums this March. As part of the multi-faceted partnership, we are also exploring ways to expand this content across the Microsoft Education portfolio.
Starting this March, we’ll be partnering with BBC Worldwide Learning to bring the stunning BBC Earth natural history film Oceans: Our Blue Planet to classrooms and museums around the world, starting with the release of the film in museums this March.
And together with LEGO® Education, we’ll be offering a new free online Hacking STEM lesson plan that has students use the Pythagorean Theorem to explore and measure topography in 2D/3D space. Acting as environmental surveyors and engineers, students build tools with LEGO® bricks from the Simple & Powered Machines set or cardboard to create and visualize an initial transportation plan for the development of an island National Park in Excel. They then bring their National Park to life by adding topographic elements.
With all of these new experiences available in time for the upcoming school year, there’s never been a better time for schools to try Microsoft Education. We will continue to work hard to deliver innovative technology for educators around the world to unlock limitless learning for students of all abilities.
*LEGO, the LEGO logo are trademarks and/or copyrights of the LEGO Group. ©2018 The LEGO Group. All rights reserved. **$219 global pricing ***If at least 30 qualified licenses are purchased between February 1, 2018, and December 31, 2018, you can schedule free in-person or virtual Professional Development led by certified Microsoft training partners. Learn more on microsoft.com/education. **** “Bringing the Universe to America’s Classrooms,” is a cooperative agreement awarded to WGBH from the National Aeronautics and Space Administration (NASA) under cooperative agreement award No. NNX16AD71A.

#ifdef WINDOWS – Getting Started with Mixed Reality

I sat down with Vlad Kolesnikov from the PAX Mixed Reality team in Windows to talk about what mixed reality is and how developers can get started building immersive experiences. Vlad covered what hardware and tools are needed, building world-scale experiences, and using the Mixed Reality Toolkit to accelerate development for Microsoft HoloLens and Windows Mixed Reality Headsets.
Check out the video above for the full overview and feel free to reach out on Twitter or in the comments below.
Happy coding!

Watching in awe as artists bring their visions to life in mixed reality

I’ll never forget the first time I experienced mixed reality.
It was a pre-production version of Microsoft HoloLens, and I tried a demo that had fish swimming around the room, seaweed on the floor and bubbles floating around me.  Eventually, the demo team had to gently suggest I leave, because I was mesmerized.  The fish were bouncing off the walls. The bubbles were popping on the ceiling.  I could see everything in the room and the people in it, and I could see more – things that felt just as real.  I couldn’t get enough.
Part of me is still in that moment because it’s what has motivated me to work as hard as I can with teams across Microsoft to build the devices, platforms, experiences and services that will enable infinite mixed reality experiences.
Everyone working in mixed reality today is a pioneer as they push forward, defining boundaries of technology, design and user experience while reinventing industries, education and communication.
Today, I want to talk about one group of pioneers who are using mixed reality to change the way we see the world: artists.
 

Since the early days of HoloLens, we have had the chance to see what happens when artists use their unique ability to see the potential of a new medium or technology. I have always been passionate about paying attention to what artists are doing with new technology platforms.  Artists, whether visual artists, media, sculpture, dance, theater or others, have both courage and the ability to see things differently.
It has been amazing to see what has been created.
Cisneros Fontanals Art Foundation Exhibit at Art Basel
Last week we kicked off a partnership with Cisneros Fontanals Art Foundation (CIFO) at Art Basel in Miami. Their exhibit, “Sandu Darie: An Immersive Experience,” aims to make important Cuban works of art accessible beyond the constraints of their physical location. The experience makes available public works of art that are permanently located in Cuba and brings them to life using Mixed Reality Viewer, Windows Mixed Reality headsets and HoloLens. For the first time outside of Cuba, visitors could view large-scale works of art in their real world through these two immersive experiences.
A photographer captures images of Cuban artist Sandu Darie’s El Día y La Noche (1982) in Havana that were used to digitally recreate the sculptural mural as a hologram
Case Western Reserve University mixed reality dance shared experience
Case Western Reserve University has truly been a pioneer in mixed reality. From the building of anatomy class experiences, to expanding to physics classes, and more recently exploring the arts with the British Museum and others, the team at Case has been leading from the edge.
I recently had the opportunity to have another once-in-a-lifetime experience: the first-ever, large-scale shared mixed reality dance performance. Eighty people all wore HoloLens to see dancers and holograms in the same performance, for the first time.  The team had no idea how to pull this off when it was conceived by the Creative Director, XX, but that’s why we’re in this space: to bring things that seem impossible into focus, and then to just figure out how to do them.
Take a look at their blog post and video to learn more about how the vision ultimately came to life.
Cornish College of the Arts
I spent two years in early HoloLens development saying, “Imagine when you go to the first holographic art gallery opening.  There will be wine and cheese, maybe some light music, but no art.  You walk in and see empty walls, empty stands. But then you put on HoloLens and the gallery is full of rich art.”
I would marvel over this with other people.
Well, this has happened, and much earlier than I expected.
Cornish College of the Arts pioneered the first exhibit in 2016, called “Through the HoloLens,” which allows students to bring to life their artistic expression in new ways through painting, dance, performance and other forms. I was fortunate to experience this firsthand with the students of Cornish, a moment I will never forget.
Gallery goers check out students’ holographic works at the Cornish College of the Arts 2016 BFA show
Artsy at the Armory
Earlier this year we partnered with online art platform Artsy and Amsterdam-based Studio Drift to create a mixed reality art installation at The Armory Show in New York.  The project was called “Concrete Storm” and was our first collaboration in a commercial art context.  It blended physical structures with holograms, which changed perspective depending on where you stood within the exhibit, allowing visitors to experience art in an entirely new way. It was fascinating to watch the typically jaded New York art crowd’s expressions the first time they put the device on; lighting up as they walked through the concrete pillars and reaching out to touch something that wasn’t really there. Proof that mixed reality can make art more accessible and interactive when it comes alive, up close and personal.
Studio Drift founders Lonneke Gordijn and Ralph Nauta view their original HoloLens work, Concrete Drift (2017), at The Armory Show in New York
I’d like to thank all working artists for your commitment to your craft and work.  And for those of you who are deeply engaged with technology, thank you again.  Your work is so very important. As we get ready for the year ahead I am eager to see what comes next.
Lorraine

Mixed Reality momentum continues in the Modern Workplace and Microsoft HoloLens expands to 29 new markets

Today, at Microsoft Future Decoded in London, over 15,000 IT and business decision makers came together to talk about current and emerging trends happening at the intersection of business and technology. I was thrilled to be there talking about our vision for mixed reality and the work we are doing to bring powerful new capabilities to the modern workplace.
We shared how technologies like Microsoft 365, Microsoft HoloLens, Windows Mixed Reality, and 3D are helping companies, Firstline Workers, and Information Workers become agents of change in the modern workplace and digital transformation.
In addition to outlining our vision for mixed reality in the modern workplace, we also announced that HoloLens is coming to 29 new markets in Europe.
Microsoft’s vision for Mixed Reality in the Modern Workplace
At Microsoft, we are on a mission to empower every person and organization on the planet to achieve more. Mixed reality has the potential to help customers and businesses across the globe do things that until now, have never been possible. Mixed reality experiences will help businesses and their employees complete crucial tasks faster, safer, more efficiently, and create new ways to connect to customers and partners.
As we think about how mixed reality will change the way we work, it’s important to acknowledge that the very nature of what it means to work is changing.
Just as the agricultural revolution led to rapid increases in agricultural productivity, or the industrial revolution ushered in entirely new manufacturing processes, the digital revolution has again redefined what it means to work.  “Work” is increasingly defined through human ingenuity, creative problem solving, and collaboration skills. This has created a demand for technology that will support these new ways of working.
The era of mixed reality will serve as a catalyst for innovations in the workplace and we expect Firstline Workers and Information Workers to benefit significantly from solutions that blend our physical and digital reality.
Firstline Workers are more than two billion people in roles that make them the first points of contact between a company and the world it serves. They are often the first to engage customers, the first to represent a company’s brand, and the first to see products and services in action. They form the backbone of many of the world’s largest industries, but they’ve been largely underserved by technology. Without them, the ambitions of many organizations could not be brought to life. Mixed reality is poised to help them work together, problem solve, and communicate in more immersive ways.
For Information Workers, mixed reality will enable collaboration and remote work both synchronously and asynchronously. They can interact more naturally with the digital world. With mixed reality workers can change the content, the people, or even the location of a meeting, in a matter of seconds. Mixed reality delivers interfaces that help workers act upon data generated from instrumented/intelligence devices, and connect seamlessly with others across physical space.
Here is a peek into what mixed reality in the modern workplace makes possible.

HoloLens coming to 29 new European markets
Since our launch in 2016 it has been inspiring to see how customers and partners around the world are using HoloLens to transform their business. HoloLens is being used every day to help people collaborate, communicate, create, and learn.
Here are a few of the industry leaders who are innovating with mixed reality.
Ford – who are reinventing the way they do product design by blending 3D holograms digitally with both clay models and physical production vehicles.
thyssenkrupp – who have found ways to dramatically decrease operational costs with remote assist and training.
Stryker – who has set out to improve the process for designing operating rooms for hospitals and surgery centers.
To address growing demand for mixed reality solutions around the world, we are thrilled to announce that HoloLens will be coming to 29 new markets, bringing the total number of HoloLens markets to 39.
We are also working to bring some of the most asked for software updates for HoloLens to our existing customers. We are committed to delivering an update to existing customers sometime early next year.
Here is a quick and easy way to see all the markets where HoloLens is available. You can learn more about HoloLens availability in your country here.

Innovation in the modern workplace is happening now
Work today has evolved from execution of routine tasks to creative problem solving. The workplace is also changing in dramatic and fundamental ways. From open and integrated physical and digital workspaces, to personalized experiences that flow across devices, teams are more mobile and globally-distributed than ever.
I am passionate about mixed reality in the modern workplace and how it helps real people solve real problems with real solutions. Some of the ways Firstline and Information Workers are using mixed reality to improve how they get work done include:
Remote Assist – Collaborating with a remote expert to accomplish real tasks, in real spaces, in real-time.
Training – Creating customized first-person perspective training manuals and step-by-step instructions on new products, processes, and equipment.
Design – Being able to plan and layout space, collaborate with others, and walk through designs in the real world without costly physical build outs.
3D in Office – Collaborating with easy 3D content integration. This is the same 3D content that we will see bridge from PC screens to full mixed reality experiences.
Mixed Reality Meetings – Team members are looking to be able to join others in real-time or asynchronously, bring in 2D and 3D content, with flexibility to change the location at any time.
Mixed Reality Data & Analytics – Information workers are gaining access to content in a heads-up, hands-free way, that is contextually and spatially relevant.
We have also worked to ensure that our mixed reality solutions meet important environmental and safety standards.
HoloLens passes basic impact tests for protective eyewear standards in North America and Europe. It has been tested and found to conform to the basic impact protection requirements of ANSI Z87.1, CSA Z94.3 and EN 166, the most common protective glass certification standards.
HoloLens has been tested and rated IP50, able to perform as dust protected, a very important scenario for many Firstline Workers.
And for head protection, we’re happy to announce a HoloLens hard hat accessory is in production and will be available for purchase early next year.
Our customers love existing Skype functionality on HoloLens; however, there are a couple of improvements they’ve been asking for. I’m excited to share that in first quarter of next year we are shipping an additional holographic remote instruction capability built on Microsoft Teams and Azure Active Directory. Our customers will now be able to work more seamlessly within their existing IT infrastructure.
Join the Era of Mixed Reality
There is so much opportunity in front of us and so many ways to join us on our mixed reality journey.
The Commercial Suite and the Development Edition for HoloLens can be purchased online through the Microsoft Store. Check the online store in your region.
With 3D in Windows 10, everyone can experience and create amazing things in a new dimension. With PC or tablet running the Windows 10 Fall Creators Update, you have the most complete collection of 3D tools and resources available.
For developers looking to build immersive experiences for Windows Mixed Reality and HoloLens, we have numerous developer resources available.
If you are looking for someone to help you develop mixed reality solutions, check out the Mixed Reality Partner Program. This is an integrated program focused on enabling and supporting digital agencies, systems integrators, and solution providers who are committed to helping others build in mixed reality.
To everyone who has been with us on our journey to date, thank you! For those of you joining now, we can’t wait to meet you. We are looking forward to what comes next.
Talk soon,
Lorraine

Black Marble uses Microsoft HoloLens to help revolutionise Crime Scene Investigation with tuServ

Black Marble, a member of Microsoft’s HoloLens Mixed Reality Partner Program (MRPP), is one of many agencies creating ground-breaking applications for HoloLens. It has seen the potential benefits that mixed reality brings to the enterprise, by blending virtual and physical worlds, enabling revolutionary experiences to improve business processes and efficiency.
Through the use of HoloLens, employees are able to transform the manner in which they communicate, create, collaborate, and explore to provide a better customer experience, improve business collaboration and much much more.

Through Microsoft’s Universal Windows Platform (UWP), Black Marble has developed an award-winning application, tuServ, that’s transforming today’s approach to digitizing modern policing. It wanted to push the boundaries of what is possible with a Universal Windows Application, and has developed an innovative proof of concept solution that gives police forces the ability to capture content at a scene of crime, in real time.
tuServ’s Scene of Crime application has been developed with operational police officers, for operational police officers. It is revolutionising how both officers out in the field, and staff in the police station, are approaching the scene of a crime. Today, evidence contamination is a primary concern for crime scene investigation teams, and transparency in the investigation process is key for maintaining public confidence in police activity. Using Microsoft HoloLens, officers in the field can place virtual markers, trace 3D objects, and gather multi-media evidence eliminating the risk of disturbing the physical evidence at a scene. The aim of tuServ is to provide crime scene investigators more accurate and tangible evidence, and virtually transports officers back to the scene of a crime. Here, markers placed, evidence gathered, and object mesh captured can be viewed as they were at the time of investigation, giving a genuine view and understanding of the scene.

However, it’s not just officers at the scene of a crime who reap the benefits of HoloLens applications. Back at the station, officers can view the captured media evidence with tuServ via Windows devices in real time or on demand, or through the tuServ HoloLens Command and Control App. This bolsters a force’s ability to collaborate even when working remotely, and enhances communication by empowering officers to not only be verbal, but also demonstrative. The application is truly redefining the ways in which police forces operate, ensuring improved data quality and the efficient use of resources. Yet still, Command and Control Units are empowered to go one step further. tuServ’s HoloLens Command and Control App gives units the possibility to go completely portable, harnessing the power of HoloLens to view officers and incidents on a map interface, giving a comprehensive overview of everything happening in the area.
Nick Lyall, Superintendent, The Bedfordshire Police says, “As a public order and firearms commander I can say that without doubt its use, through tuServ, will revolutionize policing for years to come. As a detective I can also say that its ability to scan crime scenes and create a mapped 3D version will allow for a reduction in cross-contamination issues and allow for investigators to visualize in real time the scenes of major crime.”
“The vision and innovation of Black Marble is ground-breaking, and I look forward to taking this forward of behalf of UK policing and policing worldwide.”— Nick Lyall, Superintendent, The Bedfordshire Police

Communication, collaboration and cooperation are the basis of any police operation, and through tuServ’s HoloLens Command and Control App, Command and Control units are rapidly mobilised to react to an incident at a moment’s notice. Units can assign officers to incidents, communicate with those on the ground, and build a clear and inclusive view of every component in play at a scene. Combined, these features deliver the agility that a force needs to be responsive and reactive in a high-pressure environment where every minute counts.
This is just the beginning for mixed reality in the sector, with HoloLens applications holding real potential for the future of policing and crime scene investigation. By transforming data into visual information, HoloLens is expanding the horizons for new and exciting applications that can revolutionize the ways we think and work.
Visit HoloLens.com to learn how other industries and customers are applying the transformative power of mixed reality.

Making mixed reality: a conversation with Alexandros Sigaras and Sophia Roshal

Dr. Olivier Elemento (left) alongside with his Ph.D students Neil Madhukar and Katie Gayvert, analyze medical network data (photo courtesy of the Englander Institute for Precision Medicine)
Welcome! This is Making mixed reality, a series celebrating the passionate community creating apps and experiences with Windows Mixed Reality. Here, developers, designers, artists (and more!) share how and why they got started, as well as their latest tips. We hope this series inspires you to join the community and get building!
Meeting Alexandros Sigaras and Sophia Roshal was a lot like mixed reality: a digital-physical fusion. It first happened through a flurry of tweets and emails as Alexandros, a senior research associate at Weill Cornell Medicine (WCM), and Sophia, a WCM software engineer, rapidly prototyped a Microsoft HoloLens application to achieve the Englander Institute for Precision Medicine’s “cancer moonshot,” a promise to empower better and faster cancer research, data collaboration, and accessible care. Soon after I was lucky enough to demo their project in-person. It’s now in the Windows Store as Holo Graph, an app enabling researchers to bring their own network data into the real world to explore, manipulate, and collaborate with other researchers in real-time, be they in the same room or on the other side of the planet.
Find out what makes this team tick, and how they make big data approachable with Windows Mixed Reality.
Sophia Roshal looks at a graph of medical data (photo courtesy of the Englander Institute for Precision Medicine)
Why HoloLens, and why Windows Mixed Reality?
Sophia: It’s the logical next step. You usually constantly switch from window to window [on a PC]. With HoloLens, you stay in one place. You can just point at something; you don’t have to use your mouse. It’s just so much more of a natural environment, which is great.
The best part of mixed reality for me is seeing other people try it for the first time. They are surprised how well interactions between the real world and holograms work, and are excited to see new updates. The most exciting part is to see the endless possibilities of mixed reality. From games to medical research, there are still many applications of mixed reality to explore.
Alexandros: One of the key questions we get every single time we show HoloLens to someone who is already an avid developer is, “Why HoloLens, and not a 2D screen? Why does this revolutionize our work?” The key answer behind this is simplicity, connecting these dots. The amount of high-quality data that you can parse through with holograms is significantly more than the amount of data that you could create in a table and fuse together in your brain! Tangibility and collaboration are the biggest improvements. It’s like saying the mouse and the keyboard are absolutely great, but phone touch screens are a better user interface. We treat HoloLens as a technology that allows us to go to a higher level, make things more tangible, and remove the challenges of making connections in your brain because you actually see and manipulate them.

Using @Hololens for realtime collaborative & interactive visualization on metabolomic networks @RoshalSophia @ElementoLab @ksuhre pic.twitter.com/JP60UH8Wrs
— Alex Sigaras (@AlexSigaras) August 14, 2017

Who uses Holo Graph?
Alexandros: In a nutshell, the end users for Holo Graph are computational biologists, clinicians, and oncologists. Instead of looking at “big data” in a two-dimensional structure, they immerse themselves and explore and focus on their areas of interest in 3D. There are two scenarios that we currently use with Holo Graph. One is for cancer research and genomics, and the other is metabolomics.
For cancer research, it’s for drug discovery. We want to find how specific drugs relate to specific genes. With our app, I can upload my network that has all of this correlating information, and I can explore it, manipulating and changing the ways I look at data. If I click on a hologram of a drug, I’ll see the drug’s most up-to-date information directly from the Food and Drug Administration (FDA) – that’s an API tool. If I click on the gene, gene cards will tell me more about that specific gene.
The other use case that we’re doing with our colleague Dr. Karsten Suhre from Weill Cornell Medicine in Qatar is metabolomics. Dr. Karsten Suhre has identified connections between metabolites, genes, and diseases such as Crohn’s disease and diabetes. Using Holo Graph he can browse and identify unexpected paths in the network. One of the latest videos that we shared was collaborative sharing and manipulation of this very network on Crohn’s disease to identify if there are unexpected connections to other diseases.
The Holo Graph team comes from the Englander Institute for Precision Medicine at Weill Cornell Medicine and the  Weill Cornell Medicine Qatar. From left, Sophia Roshal, Dr. Karsten Suhre, Dr. Olivier Elemento, Dr. Andrea Sboner and Alexandros Sigaras. (Photo courtesy of the Englander Institute for Precision Medicine)
When did you get started building and designing for Windows Mixed Reality? Any tips for others just beginning? 
Alexandros: We became interested on the platform about two years ago and were delighted to be included in the first wave of HoloLens devices that shipped. Whether on an online forum or a meetup, there are a lot of talented people happy to help you get there and share their experience. Don’t try to reinvent the wheel. You will be surprised how many questions have already been answered before! As far as tips go, download and try out apps from the Windows Store and make sure to reach out to the community for any questions.
Sophia: Fragments was the biggest inspiration for me because the virtual characters sit on actual chairs. We recently updated our app to include avatars and do the same thing, and it was really cool to see that! Our avatars follow the person’s movement, and we also use spatial mapping to find the floor because the only point of reference on HoloLens is the head. There are MixedRealityToolkit scripts that finds planes, where you can find the lowest one and that will be the floor. Then you calculate the height between the head and the floor and you can map the avatar from there.
Alexandros: Doing tutorials Holograms 240 with the avatars and Mixed Reality 250 with sharing across devices are excellent examples to see the capabilities here.
Sophia: For someone just beginning, Mixed Reality Academy is by far the easiest way to start building apps. MixedRealityToolkit is the main tool I use. I will write my own scripts most of the time, but if you’re just starting, you have to get it!
What inspires you?
Sophia: Impact on our patients’ lives. Seeing your code making a positive impact in someone else’s life battling with cancer is one of the most rewarding experiences ever. The Englander Institute for Precision Medicine is using cutting-edge technology to go through a sea of data and provide our care team and their patients better treatment options. We believe that AI and devices such as HoloLens are just beginning to show their true potential, and we look forward on what’s yet to come in the near future.
Alexandros: And it’s all about the power of people. The headset doesn’t save someone’s life; the clinician does. But mixed reality helps them see the patterns and get there. With HoloLens we want to answer, “If I were to show you this before you made the call, would that change anything? Would it make your decision and response faster? Would it give you more data?” And every person that we ask nods their head and says, “Yes, it’s right there. It’s almost like I can touch it.” Clinicians are used to reviewing genomic reports that can span up to hundreds of pages requiring significant time and effort. This doesn’t have to be the case though. HoloLens can act as a catalyst by significantly reducing the review time and make an impact at scale when combined with other tools such as AI, machine learning, and deep learning that we also do at the Institute.
Holo Graph can help researchers identify patterns in networks (photo courtesy of The Englander Institute for Precision Medicine)
How are you getting data into your application? Any tips for those who want to do data visualization with HoloLens?
Sophia: The easiest way to load dynamic data into an application is through a cloud integration app such as OneDrive or Dropbox. When you share data across the network to other users you need to consider secure transfer and adopt standard formats. Holo Graph currently supports .csv and XML/GraphML formats on OneDrive. We tend to share data across using JSON.
OneDrive loading has been a great surprise. We used to add files to the backend. With OneDrive, now anyone who wants to can load their data into the experience.
Alexandros: Data and data privacy are of utmost importance to the Institute. The real value of Holo Graph is not just about its looks; it’s about empowering researchers to get their real data in securely. As far as visualizing the data, my tip is to enable your users to break out of the 2D window and put their data on their environment on their terms.
I’m ending with a favorite quote from our conversation. Spoiler alert: mixed reality’s got game!
Alexandros: The way I explain mixed reality is this: Imagine you have a virtual basketball. If you’re throwing it onto the real ground, it bounces off because it knows where the ground is, and there’s “friction.” You can repeat this 100 times and it would happen the same way – you can expect it. It’s literally bringing digital content into real life, allowing you to bend the rules.
Sophia and Alexandros are seriously inspiring. You can connect with Sophia and Alex on Twitter @RoshalSophia and @AlexSigaras.
Want to get started #MakingMR? You can always find code examples, design guides, documentation, and more at Windows Mixed Reality Dev Center. Want more? Check out mixed reality design insights on Microsoft Design Medium. Inspiration abounds!

Windows Mixed Reality holiday update

We are on a mission to help empower every person and organization on the planet to achieve more, and one of the ways we are doing that is through the power of mixed reality. Since January 21, 2015, when we announced Microsoft HoloLens, we have seen developers, partners, and customers innovate in ways we have never seen before. As a creator, it is inspiring to see the world embrace mixed reality; to see organizations and developers stretch the boundaries of what we can do with technology. Together we have created the most vibrant mixed reality community out there and it has been phenomenal to share in this journey with our community.  
A little over ten months ago our mixed reality journey took a leap forward when we announced that the world’s largest PC makers were working with us to democratize virtual reality this holiday with Windows Mixed Reality headsets.  
This week at IFA we will be sharing more details on Windows Mixed Reality headsets and I encourage everyone to tune into the news that will be coming out of Berlin.  
As we get ready for a big week with our partners, I would like to share with you some exciting details about our product, including the first wave of content experiences you can immerse yourself in this holiday.
Windows Mixed Reality: easy setup, affordable gear, a range of hardware choices, and immersive experiences 
Easy setupExisting high-end VR headsets with external cameras are cumbersome to set up. For more people to experience VR one of the barriers that needed to be removed was the need for external markers. That is why the Windows Mixed Reality headsets coming this holiday will be the first to deliver VR experiences with built-in sensors to track your physical position without requiring you to purchase and install external sensors in your room. You don’t need to spend hours to set up a single room in your house with a large play space, just plug and play. This also means these experiences are portable – whether you are traveling for work or visiting friends and family, just pack your PC and headset and you can share the magic of mixed reality.  
 Affordable gear
A variety of Windows Mixed Reality headsets and motion controllers will be available this holiday from HP, Lenovo, Dell, and Acer.  Headset and motion controller bundles will start as low as $399 and will be compatible with exciting and new PC models starting at $499. Along with our partners, we are committed to making mixed reality affordable. 
Range of hardware choices
When it comes to deciding which hardware is right for you, we know that our customers value choice in brand, industrial design, and features. That is why we created Windows Mixed Reality as a platform for you to enjoy experiences across multiple devices that meet your mobility and performance needs.  
We have talked a lot about the headsets and motion controllers, now let’s talk about PCs that will be compatible with Windows Mixed Reality. This holiday, customers can choose the PC that’s right for them – Windows Mixed Reality PCs and Windows Mixed Reality Ultra PCs. Here is some context on what makes the two experiences different: 
Windows Mixed Reality PCs: will consist of desktops and laptops with integrated graphics.  When plugged into these devices, our immersive headsets will run at 60 frames per second.  
Windows Mixed Reality Ultra PCs: will consist of desktops and laptops with discrete graphics. When plugged into these devices, our immersive headsets will run at 90 frames per second.   
Both configurations will support today’s immersive video and gaming experiences such as traveling to a new country, exploring space, swimming with dolphins, or shooting zombies.  Use your Windows Mixed Reality motion controllers to enjoy a world of discovery and imagination this holiday.  
Immersive experiences
We are working with an incredible set of partners to bring the most immersive experiences to the Windows Store. First, we are excited to announce the first wave of content partners coming to Windows Mixed Reality. Second, it’s my pleasure to let you know that we are working with 343 Industries to bring future Halo experiences into mixed reality.  We are not providing specifics right now, but it is going to be a lot of fun to work with them. 

In addition, I am thrilled to announce that Steam content will also run on Windows Mixed Reality headsets. Virtual reality enthusiasts know that Steam is a great place to enjoy cutting edge immersive experiences. We can’t wait to bring their content to you.   
Here is a sneak peek into what you can expect this holiday. We are just getting started and we are honored to work with world class creators and developers.  

Mixed Reality is the future, and we want everyone on the journey with us. For customers, we are creating the best, most affordable mixed reality headsets and bringing you immersive experiences that you will love. For developers, we are making it easy to create great content spanning from simple augmented reality to virtual reality and of course holograms.  
There is more in store for this holiday and I look forward to sharing more details with you in the coming weeks! In the meantime, don’t hesitate to get in touch with me on Twitter, and please continue to share your feedback with us. 
Alex

Making mixed reality: a conversation with Lucas Rizzotto

I first met Lucas Rizzotto at a Microsoft HoloLens hackathon last December, where he and his team built a holographic advertising solution. Fast forward to August, and he’s now an award-winning mixed reality creator, technologist, and designer with two HoloLens apps in Windows Store: MyLab, a chemistry education app, and CyberSnake, a game that makes the most of spatial sound…and holographic hamburgers. Little did I know, Lucas had no idea how to code when he started. Today, he shares how he and you can learn and design mixed reality, as well as some tips for spatial sound. Dig in!

Why HoloLens, and why Windows Mixed Reality?
It’s the future! Having the opportunity to work with such an influential industry on its early days is a delightful process – not only it’s incredibly creatively challenging, you can really have a say on what digital experiences and computers will look like in 10, 20 years from now – so it’s packed with excitement, but also responsibility. We are designing the primary way most people will experience the world in the future, and the HoloLens is the closest thing we’ve got to that today.
The community of creators around this technology right now is also great – everyone involved in this space is in love with the possibilities and wants to bring their own visions of the future to light. Few things beat working with people whose primary fuel is passion.
How did you get started developing for mixed reality?
I come from mostly a design background and didn’t really know how to code until two years ago – so I started by teaching myself C# and Unity to build the foundation I’d need to make the things I really wanted to make. Having the development knowledge today really helps me understand my creations at a much deeper level, but the best part about it is how it gives me the ability to test crazy ideas really quickly and independently – which is extremely useful in a fast-paced industry like MR.
HoloLens wise, the HoloLens Slack community is a great place to be – it’s very active and full of people that’ll be more than happy to point you in the right direction, and most people involved in MR are part of the channel. Other than that, the HoloLens forums are also a good resource, especially if you want to ask questions directly to the Microsoft engineering team. Also, YouTube! It has always been my go-to for self-education. It’s how I learned Unity and how I learned a ton of the things I know about the world today. The community of teachers and learners there never ceases to amaze me.
Speaking of design, how do you design in mixed reality? Is anything different?
MR is a different beast that no one has figured out quite yet – but one of the key things I learned is that you need to give up a little bit of control in your UX process and design applications more open ended. We’re working with human senses now, and people’s preferences vary wildly from human to human. We can’t micro-manage every single aspect of the UX like we do on mobile – some users will prefer to use voice commands, others will prefer hand gestures – some users get visually overwhelmed quickly, while others thrive in the chaos. Creating experiences that can suit all borders of the spectrum is increasingly essential in the immersive space.
3D user interfaces are also a new challenge and quite a big deal in MR. Most of the UI we see in immersive experiences today (mine included!) is still full of buttons, windows, tabs and reminiscent visual metaphors from our 2D era. Cracking out new 3D metaphors that are visually engaging and more emotionally meaningful is a big part of the design process.
Also, experiment. A lot. Code up interactions that sound silly, and see what they feel like once you perform them. I try to do that even if I’m doing a serious enterprise application. Not only this is a great way to find and create wonder in everything you build, it will usually give you a bunch of new creative and design insights that you would never be able to stumble upon otherwise.
An example – recently I was building a prototype for a spiritual sequel to CyberSnake in which the player is a Cybernetic Rhinoceros, and had to decide what the main menu looked like. The traditional way to set it up would be to have a bunch of floating buttons in front of you that you can air tap to select what you want to do – but that’s a bit arbitrary, and you’re a Rhino! You don’t have fingers to air tap. So instead of pressing buttons from a distance, I made it so players are prompted to bash their head against the menu options and break it into a thousand pieces instead.
This interaction fulfills a number of roles: first of all, it’s fun, and people always smile in surprise the first time they destroy the menu it. Secondly, it introduces them to a main gameplay element (in the game players must destroy a number of structures with their head), which serves as practice. Thirdly, it’s in character! It plays into the story the app is trying to tell, and the player immediately becomes aware of what they are from that moment forward and what their goal is. With one silly idea, we went from having a bland main menu to something new that’s true to the experience and highly emotionally engaging.
HoloLens offers uniquely human inputs like gaze, gesture, and voice. So different from the clicks and taps we know today! Do you have a favorite HoloLens input?
Gazing is highly underestimated and underused – it implies user intention there’s so much you can do with it.  A healthy combination of voice, hand gestures, and gaze can make experiences incredibly smooth with contextual menus that pop in and out whenever the user stares at something meaningful. This will be even truer once eye-tracking becomes the standard in the space.
What do you want to see more of, design wise?
I want to be more surprised by the things MR experiences make me do and feel challenged by them! Most of the stuff being done today is still fairly safe – people seem to be more focused on trying to find ways to make the medium monetizable instead of discovering its true potential first. I live for being surprised, and want to see concepts and interactions that have never crossed my mind and perfectly leverage the device’s strengths in new creative ways.
Describe your process for building an app with Windows Mixed Reality.
I try to have as many playful ideas as I possibly can on a daily basis, and whenever I stumble upon something that seems feasible in the present, I think about it more carefully. I write down the specifics of the concept with excruciating detail so it can go from an abstraction into an actual, buildable product, then set the goals and challenges I’ll have to overcome to make it happen – giving myself a few reality checks on the way to make sure I’m not overestimating my abilities to finish it in the desired time span.
I then proceed to build a basic version of the product – just the essential features and the most basic functionality – here I usually get a sense if the idea works or not at a most basic level and if it’s something I’d like to continue doing. If it seems promising, then the wild experimentation phase begins. I test out new features, approach the same problem from a variety of angles, try to seize any opportunities for wonder and make sure that I know the “Why?” behind every single design decision. Keep doing this until you have a solid build to test with others, but without spending too much time on this phase, otherwise projects never get done.
In user testing, you can get a very clear view of what you have to improve, and I pay close attention to the emotional reactions of users. Whenever you see a positive reaction, write it down and see if you can intensify it even further in development. If users show negative emotional reactions, find out what’s wrong and fix it. If they’re neutral through and through, then reevaluate certain visual aspects of your app to find out how you can put a positive emotion on their face. Reiterate, polish, finish – and make a release video of it so the whole world can see it. Not everyone has access to an immersive device yet, but most people sure do have access to the internet.

CyberSnake’s audio makes players hyper-aware of where they are in the game. Can you talk about how you approached sound design? After all, spatial sound is part of what makes holograms so convincing.
Sound is as fundamental to the identity of your MR experience as anything else, and this is a relatively new idea in software development (aside from games). Developers tend not to pay too much attention to sound because it has been, for the most part, ignored in the design process of websites and mobile applications. But now we’re dealing with sensory computing and sound needs to be considered as highly as visuals for a great experience.
CyberSnake uses spatial audio in a number of useful ways – whenever user’s heads get close to their tail, for example, the tail emits an electric buzz that gets louder and louder, signaling the danger and where it’s coming from. Whenever you’re close to a burger, directional audio also reinforces the location of the collectibles and where the user should be moving their head. These bits of audio help the user move and give them a new level of spatial awareness.
Sound is an amazing way to reinforce behaviour – a general rule of thumb is to always have a sound to react to anything the user does, and make sure that the “personality” of said sound also matches the action that the user is performing thematically. If you’re approaching sound correctly, the way something looks and moves will be inseparable from the way it sounds. In the case of CyberSnake, there was some good effort to make sure that the sounds fit the visual, the music and the general aesthetic – I think it paid off!
Spending some time designing your own sounds sounds like a lot of work, but it really isn’t. Grab a midi-controller, some virtual instruments and dabble away until you find something that seems to fit the core of what you’re building. Like anything else, it all comes down to experimentation.
What’s next for you?
A number of things! I’m starting my own Mixed Reality Agency in September to continue developing MR projects that are both wondrous and useful at a larger scale. I’m also finishing my Computer Science degree this year and completing a number of immersive art side projects that you’ll soon hear about – some of which you may see at a couple of major film festivals. So stay in touch – good things are coming!
As always, I’m impressed and inspired by Lucas’s work. You can connect with Lucas on Twitter @_LucasRizzotto and his website, where you’ll find nuggets of gold like his vision for mixed reality and AI in education. And maybe even his awesome piano skills.
Learn more about building for Windows Mixed Reality at the Windows Mixed Reality Developer Center.
Lucas is right about spatial sound—it adds so much to an experience—so I asked Joe Kelly, Microsoft Audio Director working on HoloLens, for the best spatial sound how-tos. He suggests using the wealth of resources on Windows Mixed Reality Developer Center. They’re linked below—peruse and use, and share what you make with #MakingMR!
Spatial sound overview
Designing/implementing sounds
Unity implementation
Programming example video (AudioGraph)
GitHub example (XAudio2)