Category Archives: apis

Auto Added by WPeMatico

New Tools in Windows Device Portal for the Windows 10 Fall Creators Update

In the Windows 10 Fall Creators Update, Device Portal now offers several new tools from across Windows to help you location test your UWP, explore Mixed Reality, build new hardware peripherals and test your apps new installation pipeline. It’s a little bit of goodness for everyone, and we’re excited to share these with you.
If you’re not familiar with Device Portal, you can check out the blog posts below to see what other tools you can find in Device Portal, or look at the new docs.microsoft.com to learn how to enable it.
And as always, all of these tools are backed by a REST API, so that you can use it from a scripting or client application environment using the Device Portal Wrapper.
Location Based Testing
Most of us don’t have the travel budgets to test our apps across the world – but pretending to travel is almost as good!  The Location tool in Device Portal lets you easily change the location that Windows reports to apps. By tapping the “Override” check box, you can swap out the device location for whatever you set using the map or lat/long text boxes. Be sure to uncheck the box when you’re done so that your location (and timezone) come back to reality – every vacation must end…

Figure 1: The News app keeping me up to date with local headlines!
This also works for web pages in Microsoft Edge, letting you test your webpages in different parts of the world.
Some notes on what this tool can and cannot do:
This doesn’t change the locale of your PC! So the News app above still saw an EN-US user in the middle of Italy.
You may not see all apps using this location. Some programs don’t use the Windows API to determine location or have special logic (e.g. using your IP address) to determine your location.
This tool marks the PositionSource of the location data as “Default.” Some apps may check for the source and alter their behavior based on it.
Happy travels!
USB Diagnostics
This one goes out to all the hardware folks – if “HLK” or “WDK” sound familiar, you might find this handy. The USB team has updated the USBView tool to work inside Device Portal, so developers working on new hardware can have more tooling at their fingertips.
The USB Devices tool can be a bit tricky to find – head to the hamburger menu in the top right, and go to “Add tools to workspace.”  Scroll to the bottom and check the “USB Devices” box, then hit “Add.” And voila – a full view of your systems USB hubs, controllers and peripherals. The hubs and controllers expand to show individual devices using the + (plus) sign, and clicking the gear will expand to show the items properties.

Streaming App Install Debugging
The Windows 10 Creators Update added ““streaming installation” for UWP, which allows a user to launch the app before it finished downloading. In order to make this easy to test, the App Model team has added a Streaming Install Debugger tool to Device Portal. To use it, deploy an app with content groups to the device, then open the Streaming Install Debugger. In it you’ll be able to edit the states of the content groups so you can test your apps behavior as streaming install is being simulated and ensure it behaves correctly when content groups are missing.

For more details, check out Andy Liu’s blog posts about the new App Installer and Streaming Install Debugger tools.
Mixed Reality Tooling
One of the bigger splashes in the Fall Creators Update is the addition of Mixed Reality to Windows Desktop. As part of that release, we’re including a suite of tools to help developers build great Mixed Reality apps. Two of these tools may look familiar to HoloLens developers – 3D View and a Framerate counter. There’s also a new app launch option that appears when you have an immersive headset attached to your PC, which lets you launch your app in Mixed Reality.
Frame rate is an important factor in making mixed reality apps comfortable, and it’s important for developers to optimize performance to hit full frame rate on the systems they support. The Frame Rate tool in the Device Portal helps by showing developers both the frame rate of their app and of the system’s compositor.

The 3D View helps when testing your immersive headset’s interactions with the real world, displaying its position as it moves through space.

Finally, what good is tooling if you can’t actually run your app in your immersive headset? Now, when you have an immersive headset attached, the Installed Apps tool will add a button letting you launch the app in the HMD. While fully immersive apps will always run in Mixed Reality, this new button is particularly useful for 2D UWP apps (or apps that switch between 2D and immersive) when you want to test them in Mixed Reality.

As always, if you have ideas for Device Portal that would help you write or debug apps, please leave us a note on our UserVoice or upvote an existing request. If you run into bugs, please file it with us via the Feedback Hub.
Related Posts:
Using Device Portal to view debug logs for UWP
Using the App File Explorer to see your app data

Creating Materials and Lights in the Visual Layer

In today’s post, we’re going wrap up this series by combining everything we’ve learned so far and take you through the steps in creating a custom material. We also have an amazing new tool to show you that empowers anyone to design a custom material. To see how you can use these custom materials in your XAML app, be sure to check out the last two posts in this series; XAML and Visual Layer Interop, part one and part two.

The Fluent Design Language is an evolving concept, rather than a one-time design language release like MDL and MDL2. It was designed to expand and grow as Microsoft and the community of creators (developers and designers), adds to what it could be. We’ll show you that anything is possible, as designers, developers and the Windows community have common ground to share their creations and create amazing materials.
Let’s get started by first showing you how a material is created by chaining effects, then we’ll explore using the new Material Creator to easily and quickly create materials.
Creating Material with the Visual Layer
CompositionEffectBrush
Whether we’ll be using the effect in a XamlCompositionBrushBase or painting a SpriteVisual, the construction of the effect graph is the core of the approach. In order to create the material we want, we’ll need the following components:
Effect sources: A SurfaceBrush for the NormalMap and a BackdropBrush for access to the pixels underneath the material
The effect graph: A composite of different effects to control the Material’s reflectance properties and filter effects (such as blur and tint) to customize for UI usage
Lighting: The Visual is in a scene that has a CompositionLight applied
Let’s start with the first source, a SurfaceBrush. This will be provided by using LoadedImageSurface to load a NormalMap.
NormalMap and LoadedImageSurface

If you’ve had any experience with 3D computer graphics, maybe as a game developer, you may already know what a normal map is and the image above looks familiar. If you’ve never worked with one before, Normal mapping is a technique that determines the reflectance of light at every pixel (read more about Normal mapping here). The Visual Layer in Windows 10 gives you a choice of industry standard reflectance models; Blinn-Phong and Physically Based Blinn-Phong. Go here to read more about the math behind how this is done.
For today’s demo, we used a 2D picture of a textured surface and transformed it into a Normal map image using an image editor. There are many image editing tools that let you do this. You can use any one you prefer to create your image.
To get started, we can load the image using the new LoadedImageSurface API. Let’s add a private field for the LoadedImageSurface, load the image and create a CompositionSurfaceBrush with it.

// Load NormalMap onto an ICompositionSurface using LoadedImageSurface
LoadedImageSurface _surface = LoadedImageSurface.StartLoadFromUri(new Uri("ms-appx:///Images/NormalMap.jpg"), new Size(512,384));

// Load Surface onto SurfaceBrush
CompositionSurfaceBrush normalMap = compositor.CreateSurfaceBrush(_surface);
normalMap.Stretch = CompositionStretch.Uniform;

Now we’re ready to move on to creating and chaining effects.
Chaining Effects to create the effect graph
For our material, we are going to create a chain of effects that leverages the Win2D’s ArithmeticCompositeEffect (note: be sure to add the Win2D NuGet package to your project). All effects can be used as input sources for other effects, thus enabling you to allow a chain of effects to one or more inputs.
ArithmeticCompositeEffect lets you assign two sources for the composite, giving each one a weight toward the final effect. For example, Source1 at 0.75 (75%) and Source2 at 0.25 (25%). You can also use an additional ArithmeticCompositeEffect as one of the sources to add more effects in the composite chain.
Let’s step back for a minute and think about how we want create the composite:
Parent ArithmeticCompositeEffect to be used for Brush
Source 1: Child ArithmeticCompositeEffect
Source 1: ColorSourceEffect for tint coloring
Source 2: GaussianBlurEffect using the BackDropBrush for its source

Source2: SceneLightingEffect using the Normal map for its source

For source 1, we’ll combine a ColorSourceEffect and GaussianBlurEffect (from Win2D) with a nested ArithmeticSourceEffect. For Source 2, we’ll use a SceneLightingEffect (from Windows.UI.Composition.Effects). This will manipulate the reflective properties of the effect’s source when a CompositionLight, from a XamlLight for example, is applied.
Note that the SceneLightingEffect is used to modify the default lighting applied to the contents of a SpriteVisual targeted by a CompositionLight. In today’s example, we are going to create SurfaceBrush using a NormalMap (loaded by LoadedImageSurface) to define dents and bumps that the light reflects off of.
Furthermore, in order to use the SceneLightingEffect, the content being modified must be defined as one of the sources into a multi-input effect graph, with the other input being the SceneLightingEffect. For example, above, the content whose lighting properties are being modified is defined by Source1 of the parent ArithmeticCompositeEffect.
Here’s what the code looks like for the effect graph:

// Define Effect graph
const float glassLightAmount = 0.5f;
const float glassBlurAmount = 0.95f;
Color tintColor = Color.FromArgb(255, 128, 128, 128);

var graphicsEffect = new ArithmeticCompositeEffect
{
Name = "LightComposite",
Source1Amount = 1,
Source2Amount = glassLightAmount,
MultiplyAmount = 0,
// Nested Composite to combine the Blur and Color tint effects
Source1 = new ArithmeticCompositeEffect
{
Name = "BlurComposite",
Source1Amount = 1 – glassBlurAmount,
Source2Amount = glassBlurAmount,
MultiplyAmount = 0,
Source1 = new ColorSourceEffect
{
Name = "Tint",
Color = tintColor,
},
Source2 = new GaussianBlurEffect
{
BlurAmount = 20,
Source = new CompositionEffectSourceParameter("Backdrop"),
Optimization = EffectOptimization.Balanced,
BorderMode = EffectBorderMode.Hard,
},
},
// The SceneLighting effect, which will use a NormalMap
Source2 = new SceneLightingEffect
{
AmbientAmount = 0.15f,
DiffuseAmount = 1,
SpecularAmount = 0.1f,
NormalMapSource = new CompositionEffectSourceParameter("NormalMap")
}
};

Notice the SceneLightingEffect’s NormalMapSource property and the GaussianBlurEffect’s Source; these are parameter provided sources. We will set what these parameters are as we pull everything together to create the CompositionEffectBrush:

// Create EffectFactory and the CompositionEffectBrush
CompositionEffectFactory effectFactory = Window.Current.Compositor.CreateEffectFactory(graphicsEffect);
CompositionEffectBrush effectBrush = effectFactory.CreateBrush();

// Create BackdropBrush, this is used by the GaussianBlurEffect
CompositionBackdropBrush backdrop = Window.Current.Compositor.CreateBackdropBrush();

// Set Sources to Effect
effectBrush.SetSourceParameter("NormalMap", _normalMap);
effectBrush.SetSourceParameter("Backdrop", backdrop);

With the CompositionEffect completed, we can now use it to paint a SpriteVisual, like this:

SpriteVisual spriteVisual = Window.Current.Compositor.CreateSpriteVisual();
spriteVisual.Brush = effectBrush;

If you’re primarily a XAML dev, you can use this effect in a XamlCompositionBrushBase. Let’s take a look.
Using the CompositionEffectBrush in a XamlCompositionBrushBase
As I mentioned earlier, we can also create this effect graph in XamlCompositionBrushBase and set the XamlCompositionBrushBase’s CompsositionBrush property. If you haven’t read the post in this series on how to create a XamlCompositionBrushBase, go here to catch up.
As with the other XamlCompositionBrushBase implementations, we build the effect graph in the OnConnected method and make sure that the user’s device supports effects. This can be done using the AreEffectsSupported method of the CompositionCapabilities API.
Here’s the full class:

public sealed class MaterialBrush : XamlCompositionBrushBase
{
private LoadedImageSurface _surface;

protected override void OnConnected()
{
if (DesignMode.DesignModeEnabled) return;

Compositor compositor = Window.Current.Compositor;

// CompositionCapabilities: Are Effects supported?
bool usingFallback = !CompositionCapabilities.GetForCurrentView().AreEffectsSupported();
FallbackColor = Color.FromArgb(100, 60, 60, 60);

if (usingFallback)
{
// If Effects are not supported, use Fallback Solid Color
CompositionBrush = compositor.CreateColorBrush(FallbackColor);
return;
}

// Load NormalMap onto an ICompositionSurface using LoadedImageSurface
_surface = LoadedImageSurface.StartLoadFromUri(new Uri("ms-appx:///Images/NormalMap.jpg"), new Size(512, 384));

// Load Surface onto SurfaceBrush
CompositionSurfaceBrush normalMap = compositor.CreateSurfaceBrush(_surface);
normalMap.Stretch = CompositionStretch.Uniform;

// Define Effect graph
const float glassLightAmount = 0.5f;
const float glassBlurAmount = 0.95f;
Color tintColor = Color.FromArgb(255, 128, 128, 128);

var graphicsEffect = new ArithmeticCompositeEffect()
{
Name = "LightComposite",
Source1Amount = 1,
Source2Amount = glassLightAmount,
MultiplyAmount = 0,
Source1 = new ArithmeticCompositeEffect()
{
Name = "BlurComposite",
Source1Amount = 1 – glassBlurAmount,
Source2Amount = glassBlurAmount,
MultiplyAmount = 0,
Source1 = new ColorSourceEffect()
{
Name = "Tint",
Color = tintColor,
},
Source2 = new GaussianBlurEffect()
{
BlurAmount = 20,
Source = new CompositionEffectSourceParameter("Backdrop"),
Optimization = EffectOptimization.Balanced,
BorderMode = EffectBorderMode.Hard,
},
},
Source2 = new SceneLightingEffect()
{
AmbientAmount = 0.15f,
DiffuseAmount = 1,
SpecularAmount = 0.1f,
NormalMapSource = new CompositionEffectSourceParameter("NormalMap")
},
};

// Create EffectFactory and EffectBrush
CompositionEffectFactory effectFactory = compositor.CreateEffectFactory(graphicsEffect);
CompositionEffectBrush effectBrush = effectFactory.CreateBrush();

// Create BackdropBrush
CompositionBackdropBrush backdrop = compositor.CreateBackdropBrush();

// Set Sources to Effect
effectBrush.SetSourceParameter("NormalMap", normalMap);
effectBrush.SetSourceParameter("Backdrop", backdrop);

// Set EffectBrush as the brush that XamlCompBrushBase paints onto Xaml UIElement
CompositionBrush = effectBrush;
}

protected override void OnDisconnected()
{
// Clean up resources
_surface?.Dispose();
_surface = null;

CompositionBrush?.Dispose();
CompositionBrush = null;
}
}

To see this in action, let’s create a Grid to put in the middle of our page’s root Grid and set an instance of our MaterialBrush to that Grid’s Background brush:

<Grid Background="Gray">
<Grid Width="580"
Height="387"
HorizontalAlignment="Center"
VerticalAlignment="Center">

<!– Our new MaterialBrush –>
<Grid.Background>
<brushes:MaterialBrush />
</Grid.Background>
</Grid>
</Grid>

Here’s what it would look like if you ran the app now:

This is because you’re missing the second part of the approach, the lighting!
Illuminating the Material with Lights
In the last post, we created two lights (an AmbientLight “AmbLight” and the SpotLight “HoverLight”). We’ll use them today to apply lighting to the UIElement that is using our custom material.
Since our MaterialBrush uses the new SceneLightingEffect with a Normal map, any lights applied will enhance the material per the SceneLightingEffect’s configuration. Note that this isn’t necessary, but can greatly enhance your material. For example, if you’re using an Acrylic material in your app, adding Reveal will enhance the Acrylic.
Let’s now add the two XamlLights to the Grid:

<Grid Background="Gray">
<Grid Width="580"
Height="387"
HorizontalAlignment="Center"
VerticalAlignment="Center">
<Grid.Background>
<brushes:MaterialBrush />
</Grid.Background>

<!– Added lights –>
<Grid.Lights>
<lights:HoverLight />
<lights:AmbLight />
</Grid.Lights>
</Grid>
</Grid>

Now, this is what you’ll see at runtime:

What if it were easier to create and experiment with new materials? What if there were a tool that anyone can use? Let’s take a look at what’s coming to the WindowsUIDevLabs GitHub repo, the Material Creator tool.
Using the new Material Creator
Introducing availability of the new Material Creator tool!

Creating custom materials may sometimes requires a bit of experimentation and tweaking to get the effect’s property configuration just right. This would take time if you had to constantly tweak and redeploy your app. What if there were a way that you could change effect properties and material layers in real time?
The Material Creator can be found on the WindowsUIDevLabs GitHub repo in the demos folder here. (Note: you need to be running Windows 10, build 16225 or higher to use the Material Creator).
Generating the SceneLightingEffect code

One of the great features of the tool is being able to see the effect graph after you’re done creating the material. Click the ellipsis next to the save button and select “view effect graph” to see the C# code for the SceneLightingEffect. You can then copy and use this code directly in your custom material class.
If you go back up to the part of this article where we created the ArithemticCompositeEffect that contains a SceneLightingEffect, that’s where you can use this code!
Saving and Loading Materials
The Material Editor can also save and load materials! If you want to save your current progress on a material, or share a completed material with another developer, just click the Save button and it will create a json file containing all the layers and effect configurations. To load an existing material or edit a material shared with you, just click Load and select the json file.
The key takeaway is that you don’t need to be a developer to create materials. A designer can create a material and then share the saved json file with a developer for implementation in a XamlCompositionBrushBase. Even Windows enthusiasts, like the Windows Insiders, can start building out a universe of materials to drive the evolution of Fluent Design.
Blog Series Wrap up: The future of Fluent Design materials
Acrylic and Reveal are stunning examples of how using Material with Lights can alter the Windows experience, but they’re just the beginning of what is possible. The vision for the Fluent Design Language going forward is that developers and designers can easily build custom materials, innovate and share as well.
The message we want you to walk away with is that you can build new Materials, for a couple primary reasons:
Because you’re a Creator
Because it embodies your brand
We look forward to seeing what kinds of materials you create for your Windows apps! If you’ve already built your own material, feel free to share in the comments below.
Resources
Demo Code:
Material Creator
Brush Interop
Light Interop

Blog posts: XAML and Visual Layer Interop series
Part One
Part Two

Lighting Overview
CompositionCapabilities API
Light Types
Mathematics of lighting
SceneLightingEffect
WindowsUIDevLabs GitHub repo (contains Samples Gallery app)
Win2D Effects API Documentation

Powered by WPeMatico