Tag Archives: adaptive ui

UWP Hosted Web App on Xbox One (App Dev on Xbox series)

For the fourth installment in the series, we are open sourcing yet another sample app: South Ridge Video, an open source video app developed as a hosted web application built with React.js and hosted on a web server. South Ridge can easily be converted to a UWP application that takes advantage of native platform capabilities and can be distributed through the Windows Store as any other UWP app. The source code is available on GitHub right now, so make sure to check it out.

If you missed the previous blog post from last week on Background Audio and Cross Platform Development with Xamarin, make sure to check it out. We covered how to build a cross-device music experience using Xamarin and how to support background audio on the Universal Windows Platform, including the Xbox One. To read the other blog posts and watch the recordings from the App Dev on Xbox live event that started it all, visit the App Dev on Xbox landing page.

South Ridge Video

image1

What if you could extend the investment put into your web application and make it available as an Xbox One Windows Store app? What if you could also continue to use your existing web frameworks, CDN and server backend, yet still be able to use native Windows 10 APIs?

You can! A Hosted Web App (HWA) is a web app that can be submitted to the Store just like any other Universal Windows Platform (UWP) app. Since you’ve already invested in the development of the web app, your transformation into a HWA can be done rather quickly and easily.

To do this, you’ll reuse your existing code and frameworks as if you were developing for the browser. Let’s take a look at the South Ridge Video example app on GitHub. This is an app built with React.js and uses a web server back-end.  The application is delivered by the server when the app is launched and the UWP app will host that web content.

However, the HWA is not just a simple wrapper; you can also call into native Windows 10 APIs, such as adding an event to the user’s calendar. You can even use APIs such as toast notifications and camera capture or media transport controls with background audio. Note that when accessing sensitive APIs, make sure you declare the required App Capability in the appmanifest.

Now that your web app is ready and you want to turn it into a HWA, let’s take a look at the things you need to do to get it ready for Windows Store distribution to Xbox One.

Local Testing on a retail Xbox One

First, you need to put your retail Xbox One into developer mode so you can deploy and test the app. This is pretty straightforward; it involves installing an app from the Windows Store. Go here to see the steps on how to activate dev mode and how it works.

Now that you have your Xbox ready for deployment, let’s review the ways you can generate the app. There are three main options:

  • Using a web browser-based tool, such as Windows App Studio, to generate it for you
  • On a Mac or PC using Manifold.js
  • On a PC using Visual Studio

Let’s drill down into each option to see which best fits your needs.

Windows App Studio

Let’s start with the browser-based option, using Windows App Studio to create the HWA. Windows App Studio is a free online app creation tool that allows you to quickly build Windows 10 apps.

  1. Open Windows App Studio in your web browser.
  2. Click Start now!
  3. Under Web app templates, click Hosted Web App.
  4. Follow the on-screen instructions to generate a package ready for publishing to the Windows Store.
  5. You can then download the generate package that you’ll publish to the Store.

On a Mac using Manifold.js

What you need:

  • A web browser
  • A command prompt

ManifoldJS is a Node.js app that easily installs from NPM. It takes the meta-data about your web site and generates native hosted apps across Android, iOS and Windows. If your site does not have a web app manifest, one will be automatically generated for you. For example, take a look at the web app manifest for South Ridge Video.

  1. Install NodeJS, which includes NPM (Node Package Manager)
  2. Open a command prompt and type “NPM install -g manifoldJS”
    1. npm install -g manifoldjs
  3. Run themanifoldjs command on your web site URL:
    1. manifoldjs http://southridge.azurewebsites.net
  4. Follow the steps in the video below to complete the packaging (and publish your Hosted Web App to the Windows Store)

image2

A couple notes about using Manifold.js

  • If you have a W3C manifest, it will use that for app info; otherwise, it will be created automatically.
  • Use the -windows10 platform flag to generate only for UWP.

On a Windows PC using Visual Studio

What you need:

  • Visual Studio 2015. The free, full-featured Visual Studio Community 2015 includes Windows 10 developer tools, universal app templates, a code editor, a powerful debugger, Windows Mobile emulators, rich language support and much more—all ready to use in production. The same is true for Professional or Enterprise variants of VS 2015.
  • (Optional) Windows Standalone SDK for Windows 10. If you are using a development environment other than Visual Studio 2015, you can download a standalone Windows SDK for Windows 10 installer. Note that you do not need to install this SDK if you’re using Visual Studio 2015 Update 3; it is already included.

Steps to create the HWA (go here to see all these steps with screenshots):

  • Pick the website URL and copy it into your clipboard.
  • Launch Visual Studio 2015.
    1. Click File.
    2. Click New Project.
    3. Under JavaScriptthen Windows Universal, click Blank App (Windows Universal).
  • Delete the VS project template generated code.
    1. Since this is a hosted web app where the content is served from a remote server, you will not need most of the local app files that come with the JavaScript template by default. Delete any local HTML, JavaScript, or CSS resources. All that should remain is theappxmanifest file, where you configure the app and the image resources.
  • Open the appxmanifestfile.
    1. Under the Application tab, find the Start page text field.
    2. Replace html with your website URL.
  • Set the boundaries of your app.
    1. Application Content URI Rules (ACURs) specify which remote URLs are allowed access to your app and to the Universal Windows APIs. At the very minimum, you will need to add an ACUR for your start page and any web resources utilized by that page. For more information on ACURs, click here.
    2. Open theappxmanifest file.
    3. Click theContent URIs.
    4. Add any necessary URIs for your start page.

For example:

  1. 1. http://southridge.azurewebsites.net2. http://*.azurewebsites.net
  2. Set theWinRT Access to All (for each URI you added).
  • At this point, you have a fully functioning Windows 10 app capable of accessing Universal Windows APIs! You can now deploy to the Xbox One using the remote debugger as if it were any Windows 10 remote device, or you can use the Device Portal (covered in the next section).

Installing your HWA on the Xbox One for local testing

At this point, you now have an app package (APPX) or a manifest folder (Manifold.js), containing the files you need to install or publish your application. You can “side-load” your HWA to the Xbox by using the Device Portal (go here for more information about the Device Portal for Xbox One). One you’ve logged into the portal, you can then deploy your app.

APPX Deployment (Visual Studio / App Studio)

Here are the steps (go here to see these steps below with screenshots):

  1. Go to the Apps tab on the portal.
  2. You’ll see two buttons to upload items to the device: App Package and Dependencies.
  3. Tap the App Package button and navigate to the package folder you generated for your app. In there, you’ll find an appx file. Select that file and upload it.
  4. Now tap the Dependencies button, navigate again to your package folder and drill down to the dependencies sub folder. This contains the dependencies that your app needs to run – upload each one in the folder. (Note that you only need to do this when deploying via the Portal. Visual Studio delivers dependencies when remote debugging and the end user’s machine will already have these installed when delivered via the Store.)
  5. With the app package and the dependencies uploaded, click the Go button under the “Deploy” section and the app will be installed.
  6. Now go to the Xbox and launch the app!

Loose Files Deployment (Manifold.js)

The only difference between deploying an app packages with Manifold.js is that you have “loose” files instead of an APPX package. In the steps above, instead of uploading an APPX and Dependencies, you choose the “upload loose files” option and select the manifest folder. The portal will look in that folder for the manifest file and gather all the required pieces to complete the installation.

Design and User Experience Considerations for Xbox One apps

Designing for the “10-Foot Experience”

Xbox One is considered a “10-foot experience”, meaning that your users will likely be sitting a minimum of 10 feet away from the screen. You’ll want to consider the app’s ability to be used at that distance as opposed to being accessed within a web browser, from two feet away, with a mouse and keyboard. This article explains how to make sure you get the best user experience for the 10-foot experience scenario.

Designing for TV and understanding the “TV SafeZone”

Television manufacturers will apply a “safe-zone” around the content that can clip your app. By default, we apply a safe border around your app to account for this, however you can ensure that your app takes the full screen size using following code:


var applicationView = Windows.UI.ViewManagement.ApplicationView.getForCurrentView();
applicationView.setDesiredBoundsMode(Windows.UI.ViewManagement.ApplicationViewBoundsMode.useCoreWindow);

Understanding and managing XY focus and navigation

You’ll want to consider your app’s ability to handle XY navigation by the user and disable the “Mouse Mode” that’s turned on by default for UWP apps. This is important because users’ main input method for Xbox One is a handheld controller. See here for more on how to work with XY focus and navigation. Use the following code to enable XY navigation using JavaScript:


navigator.gamepadInputEmulation = "keyboard";

To enable directional navigation, you can use the TVJS library which is discussed below.

Considering your app’s appearance when another app is snapped

When users run an app on Xbox One, a second app may be ‘snapped’ to the right of the main app. When this is the case, the main app is considered to be in Fill Mode. While testing your app, open Cortana or another ‘snappable’ app to see how your app appears. You want to make sure your UI is still usable and has a graceful transition between Full Screen and Fill Mode. Implement an adaptive UI to make sure the user has the best experience for this scenario.

Integrate with the System Media Controls

If your app is a media app, it is important that your app responds to the media controls initiated by the user via the on-screen buttons, Cortana (typically through speech), the System Media Transport Controls in the nav pane or the Xbox and SmartGlass apps on other devices. Take a look at the MediaPlayer control from TVJS which automatically integrates with these controls or check out how to manually integrate with the System Media Transport Controls.

TVJS

TVJS is a collection of helper libraries that make it easier to build web applications for the TV. If you are building a hosted web app that will also run on the Xbox, TVJS can help add support for Directional navigation, as well as provide several controls that make it easier to interact with content on the TV.

DirectionalNavigation

Directional navigation is a feature that provides automatic two-dimensional navigation within the pages of your TV app. Apps won’t need to trap and handle navigation within their pages, or to explicitly specify all the valid focus targets for each element in the UI. With automatic focus handling, users can navigate around in an intuitive and robust way.

When users enter directional input to move from one element to another, the automatic focus algorithm looks at the set of potential focus targets, determines the next element to move to and then automatically sets the focus to that element. To determine the element to move to, the algorithm combines directional input, past focus history, and the physical layout of UI elements on the page.

To enable directional navigation, include the following script reference:

<script src=”directionalnavigation-1.0.0.0.js”></script>

By default, only a, button, input, select, and text area elements are considered focus-able. To make other elements focus-able, set a valid tab index on the element.

<div tabindex=”0″>This div is eligible for focus</div>

Make sure to check out the documentation to learn how to change the root element, set initial focus, how to override next focus, best ways to optimize controls for focus and how to customize the input. Don’t miss the samples as well that go through a lot of great examples.

Submitting your app to the Windows Store

Once you’re happy with the app after testing it on Xbox, it’s time to publish it to the Windows Store!

Depending on which route you took to build the app, the process is a little different on how you build the package for Store submission.

Xbox One and Hosted Web Apps are a great way to deliver your web application to millions of Xbox One users and enter the living room experience with minimal effort.

Wrap up

Make sure to check out the app source on our official GitHub repository, read through some of the resources provided, watch the event if you missed it and let us know what you think through the comments below or on twitter.

Next time we will release another app experience and go in-depth on how to build great Internet of Things experiences using the Universal Windows Platform and how to make them shine on the Xbox One.

Until then, happy coding!

Resources for Hosted Web Apps

Download Visual Studio to get started!

The Windows team would love to hear your feedback.  Please keep the feedback coming using our Windows Developer UserVoice site. If you have a direct bug, please use the Windows Feedback tool built directly into Windows 10.

— Written by Jeff Burtoft, Principle Program Manager WDG/PAX

Designing for Intuitive Navigation

In the past several posts, we have covered user interface (UI) design concepts such as typography, visual cues, and design thinking. In today’s blog, however, we will look at a topic that has much less to do with solving UI design problems and much more to do with solving user experience (UX) design problems—that is, navigation.

Navigation involves structuring your app in a way that promotes ease and efficiency, not merely design aesthetics. Unlike the other topics covered so far, which rely in part on personal taste as well as an awareness of changing fashions, navigation—and UX in general—tends to rely heavily on usability research.

In this post, we will explore the three key principles that both research and experience have shown underpin good navigation design:

  • Consistency — meet user expectations
  • Simplicity — don’t do more than you need to
  • Clean interaction — keep out of your users’ way

Keeping these principles in mind as you design will help you to achieve the ideal navigational structure—that is, one that the user never notices.

Consistency

Navigation should be consistent with user expectations, leaning on standard conventions for icons, location and styling.

For example, in Figure 1 below, you can see the spots where users will typically expect to find functionality like the menu and back buttons. The user expectation that the menu button will be at the top-left of the screen is so strong that you can even consider using a non-traditional icon to represent it, although the traditional “hamburger” icon is often the preferred choice across most platforms. For the back button, it is better to stick with Windows convention and keep it in either the leftmost spot or, if there is a Windows button, in the second-to-the-left spot.

IMAGE1

Figure 1. Users expect to find certain buttons in certain places—e.g. the menu in the top left, and for UWP apps, the back button in the leftmost or second-to-the-leftmost spot. Sticking to these standard conventions helps users interpret the meaning of the buttons.

Placement of navigational elements should also change for different device families. For example, on tablets and laptops/PCs, the navigation pane is usually placed on the left side, whereas on mobile, it is on the top.

IMAGE2

Figure 2. Different device families have their own conventions for navigational elements. For example, the navigation pane typically appears on the left side of the screen for tablets, but up top for mobile devices.

Simplicity

Another important factor in navigation design is the Hick-Hyman Law, often cited in relation to navigational options. This law encourages us to add fewer options to the menu (see Figure 3 below). The more options there are, the slower user interactions with them will be, particularly when users are exploring a new app. The same law can be applied to media content. Rather than overwhelming the user with a vast selection of media options, consider providing brief tidbits for users to explore if they choose.

IMAGE3

Figure 3. On the left, notice there are fewer options for the user to select, whereas on the right, there are several. The Hick-Hyman Law indicates that the menu on the left will be easier for users to understand and utilize.

Clean interaction

The final key characteristic of navigation is clean interaction, which refers to the physical way that users interact with navigation across a variety of contexts.

This is one area where really putting yourself in the user’s position will inform your design. Try to understand your users and their behaviors. If you’re designing a cooking app, and you’re expecting it to be used in the kitchen, you might want to take into account that the user will probably want to avoid using food-covered fingertips to navigate to the next cooking step. Instead, the user might use a knuckle, the back of his or her hand, or even an elbow. This should influence the size of your touch targets and the appropriate spacing between navigational elements, at the very least.

You should also keep in mind which areas of the screen are considered easy to reach. These are known as interaction areas. In the mobile device illustration below (Figure 4), for example, the blue area represents the optimal touch points for users (in this case, a user with the phone held in her left hand). Here, users expend the least amount of effort to interact —remember that most users hold their phones with their left hands and interact with their thumbs. Correspondingly, the dark grey region requires somewhat greater effort for interaction than the blue, and the light gray area requires the greatest amount of effort overall.

IMAGE4

Figure 4. Mobile device interaction area.

Tablet devices introduce an additional complexity because users have multiple ways of holding their device. Typically, users grip a tablet with both hands along the sides. Figure 5 below shows the interaction area for the most common pose and grip of a tablet. Keep in mind as you design your navigation, however, that tablet users often switch between posing their devices in landscape and portrait orientations. Finally, note the alternative ways you yourself interact with tablets and consider whether your navigation is convenient for those scenarios, as well.

IMAGE5

Figure 5. Tablet device interaction area.

Rules of Thumb

Several rules of thumb help designers to encapsulate consistency, simplicity and clean interaction in their navigation design. Most of these come from the web design world and have been around for nearly a decade. These rules happen to work well for UWP apps, but as with any rules of thumb, use them as starting points and tweak as needed.

  1. Avoid deep navigational hierarchies. How many levels of navigation are best for your users? A top-level navigation and one level beneath it is usually plenty. If you go beyond three levels of navigation, then you break the principle of simplicity. Even worse, you risk stranding your user in a deep hierarchy that they will have difficulty leaving.
  1. Avoid too many navigational options. Three to six navigation elements per level seem about right. If your navigation needs more than this, especially at the top level of your hierarchy, then you might consider splitting your app into multiple apps, since you may be trying to do too much in one place. Too many navigation elements usually lead to inconsistent and unrelated objectives in your app.
  1. Avoid pogo-sticking. Pogo-sticking occurs when there is related content, but navigating to it requires the user to go up a level and then down again. Pogo-sticking violates the principle of clean interaction by requiring unnecessary clicks or interactions to achieve an obvious goal—in this case, looking at related content in a series. (The exception to this rule is in search and browse, where pogo-sticking may be the only way to provide the diversity and depth required.)
IMAGE6

Figure 6. Pogo-sticking to navigate through an app—the user has to go back (green back arrow) to the main page in order to navigate to the “Projects” tab.

Having an icon (swipe gesture in green) helps to resolve this issue, as you can see in Figure 7.

IMAGE7

Figure 7. You can resolve some pogo-sticking issues with an icon (note the swipe gesture in green).

  1. Mind your thumbs. Your users will typically use their thumbs to navigate your app. At the same time, they are also using their thumbs to grip their device, leading to a bit of clumsiness in interaction. To throw an extra wrench into this, thumbs are relatively big compared to other fingers and tend to hide the visual element with which a user is trying to interact. The technical term for this occurrence is occlusion. Keep occlusion in mind as you create your navigation structure.

Wrapping up

The goal of navigation design is to help your user move through your app without noticing your navigation structure at all. This is accomplished by making your navigation design simple and clean, typically through the reuse of the navigation idioms that everyone else uses whenever you can. By making your navigation uninteresting and consistent with standard conventions, you are actually helping your users to navigate your app intuitively.

For more information, refer to Navigation design basics for UWP apps and Navigation in UWP apps.

NPR One on the Universal Windows Platform

Earlier this year, NPR released a Windows 8 version of NPR One for Windows, and recently, a Universal Windows Platform (UWP) version. They were eager to share their challenges and developer experiences in bringing the app to UWP.

NPR One provides talk radio personalized for each listener. NPR’s mission is to provide public radio content wherever the user is: on-the-go, at home, or at work. Windows provided a new way for their growing audience to listen. “When we released the Windows app, we had an unanticipated spike in iOS and Android downloads, as well” said Ben Schein, Product Manager at NPR. “Seventy percent of NPR listeners use Windows devices, and we’ve seen a 50-percent increase in their listening time just since Windows 10 came out. And that was before we upgraded our app.” The number of users and the adoption rate of Windows 10, over 110 million in the first two months, inspired NPR to create a native Windows 10 app. In consideration of this growth, NPR collaborated with InfernoRed Technology, who had worked on the original Windows 8 app. Together, they developed the upgrade, adapted the app to use NPR’s newly updated API’s, and designed the user interface pictured below.

1_nprTheSalt
A screengrab from The Salt on NPR One

Designing NPR One

The challenge NPR originally faced with the bringing the NPR experience to the PC was upsizing their mobile app design. As they approached the PC, they needed to design for a wider variety of screen sizes without sacrificing the core app experience, which is to sit back and listen. The original NPR app experience was designed for a four to five-inch phone screen. For larger screens, they began to explore how to use the extra space to deepen the experience with images. They wanted to avoid the traditional music-related imagery approach, as they don’t want to be in the business of finding quality visuals for every story, and even considered researching an automated way to provide related images. For NPR’s update to UWP, the developers leveraged Windows 10’s adaptive layouts to dynamically change the UI in response to different screen or window sizes. For example, when the app display reaches the large screen breakpoint, the app adjusts the interface elements to make better use of the screen real estate. When it reaches the small or mobile breakpoint, the app responds with a completely different UI design tailored to the condensed screen or window size.

“We’ve always strived to provide a simple, clean experience for users that doesn’t get in the way of them listening and getting to the content they want,” Schein notes. “Once we pivoted to think about that through the scope of displays as large as 20-30 inches, it gave us an opportunity to think about ‘simple’ through a different lens.”

While they are still looking for a better solution, they found a way to make a great user experience for their listeners by displaying a high-quality image when one is available, and a placeholder image when one is not. An example of their solution is below, the story metadata is displayed, non-intrusively inviting the listener to learn more.

2_nprAllTechConsidered
A screengrab from All Tech Considered on NPR One

Using Background Audio

To create a first-class audio experience for NPR One, the app needed to provide a continuous, reliable audio stream. The dev team needed to run the audio task in the background to optimize performance, even on less-than-ideal hardware. Plus, running the logic in the background meant the UI could be lightweight, while the background task did all the heavy lifting.

The developers leveraged the Windows background audio task, which allows audio to continue playing and to be accessed through the system volume controls even after the foreground app has been closed. Previously in their Windows 8.1 apps for PC and desktop, their background audio code could not be shared between projects, but now with apps built on the UWP it can be shared by both phone and desktop device families. The playback task uses IBackgroundTask, but slightly specialized for background audio. For example, unlike traditional background tasks, it is not registered in the foreground. Through BackgroundMediaPlayer.Current, the app accesses the MediaPlayer class, part of the Windows.Media.Playback namespace.

Cortana in NPR One

As the NPR One app is a mostly hands-off experience, NPR was intrigued by Cortana and the idea of being able to use your voice in situations where it might not be practical to use your hands. The InfernoRed development team needed to make the Cortana background task communicate with the background audio task, something it was not originally designed to do. “This meant we had to code our own clever custom solution for processing commands in the voice background task, then responding and taking action on them in the audio background task,” said Ed Snider, Senior Developer at InfernoRed. Based on the results of user research, they created a set of commands that cover how most listeners interact with the audio playback. Users can play specific programs or podcasts, and even mark the current or previous story they heard as interesting. And because the app is using the background audio task, the app is able respond to default media commands using Cortana, as well, such as “Pause,” “Play,” or “Skip.”

The developers have plans to utilize Cortana further as the feature set matures. “Working with Cortana was a lot of fun. It’s really cool to be able to lay out the voice command definitions and use them as the conduit from the user to the app,” said Scott Lock, CEO and Co-Founder at InfernoRed.

3_nprDeceptiveCadence
A screengrab from Deceptive Cadence on NPR One

Part of the appeal of the app is its personalization feature and how that follows the user from one listening session to another, even to other devices. NPR One’s Windows app allows users to login to NPR One with their Microsoft Account. By leveraging the Live SDK REST API, the app provides seamless access to stored personalization settings. For example, while listening to a particular program, the users can tell Cortana to mark that content as interesting. This interaction is evaluated in the NPR One back end, which then provides listening recommendations from NPR’s curated inventory. This is done through an API call, so they won’t lose their settings or preferences, even if they change platforms.

NPR One for Windows 10 is now available in the Windows Store. If you would like to develop an app for NPR One, take a look at NPR One API and the documentation offered there.

Further reading: