Tag Archives: Visual Layer

New Lights and PropertySet Interop – XAML and Visual Layer Interop, Part Two

In the last post, we explored using XamlCompositionBrushBase and LoadedImageSurface to create custom CompositionBrushes that can be used to paint XAML elements directly in your markup. In today’s post, we’ll continue our look into the new improvements made to the XAML and Visual Layer Interop APIs available in the Windows 10 Creators Update.

In this blog series, we’ll cover some of these improvements in the Creators Update and look at the following APIs:

  • In Part 1:
    • XamlCompositionBrushBase – easily paint a XAML UIElement with a CompositionBrush
    • LoadedImageSurface – load an image easily and use with Composition APIs
  • In Part 2, today’s post:
    • XamlLights – apply lights to your XAML UI with a single line of XAML
    • PointerPositionPropertySet – create 60 FPS animations using pointer position, off the UI thread!
    • Enabling the Translation property – animate a XAML UI Element using Composition animation

If you’d like to review the previously available ElementCompositionPreview APIs, for example working with “hand-in” and “hand-out” Visuals, you can quickly catch up here.

Lighting UI with XamlLights

A powerful new feature in the Creators Update is the ability to set and use a Lighting effect directly in XAML by leveraging the abstract XamlLight class.

Creating a XamlLight starts just like a XamlCompositionBrushBase does, with an OnConnected and OnDisconnected method (see Part One here), but this time you inherit from the XamlLight subclass to create your own unique lighting that can be used directly in XAML. Microsoft uses this with the Reveal effect that comes with the Creators Update.

To see this in action, let’s build a demo that creates the animated GIF you see at the top of this post. It combines everything you learned about XamlCompositionBrushBase and LoadedImageSurface in the last post, but this example has two XamlLights: a HoverLight and an AmbientLight.

Let’s begin with creating the AmbientLight first. To get started, we begin similarly to the XamlCompositionBrushBase with an OnConnected and OnDisconnected method. However, for a XamlLight we set the CompositionLight property of the XamlLight subclass.


public class AmbLight : XamlLight
{
    protected override void OnConnected(UIElement newElement)
    {
        Compositor compositor = Window.Current.Compositor;

        // Create AmbientLight and set its properties
        AmbientLight ambientLight = compositor.CreateAmbientLight();
        ambientLight.Color = Colors.White;

        // Associate CompositionLight with XamlLight
        CompositionLight = ambientLight;

        // Add UIElement to the Light's Targets
        AmbLight.AddTargetElement(GetId(), newElement);
    }

    protected override void OnDisconnected(UIElement oldElement)
    {
        // Dispose Light when it is removed from the tree
        AmbLight.RemoveTargetElement(GetId(), oldElement);
        CompositionLight.Dispose();
    }

    protected override string GetId() => typeof(AmbLight).FullName;
}

With ambient lighting done, let’s build the SpotLight XamlLight. One of the main things we want the SpotLight to do is follow the user’s pointer. To do this, we can now use GetPointerPositionPropertySet method of ElementCompositionPreview to get a CompositionPropertySet we can use with a Composition ExpressionAnimation (PointerPositionPropertySet is explained in more detail in the PropertySets section below).

Here is the finished XamlLight implementation that creates that animated spotlight. Read the code comments to see the main parts of the effects, particularly how the resting position and animated offset position are used to create the lighting.


public class HoverLight : XamlLight 
{
    private ExpressionAnimation _lightPositionExpression;
    private Vector3KeyFrameAnimation _offsetAnimation;

    protected override void OnConnected(UIElement targetElement)
    {
        Compositor compositor = Window.Current.Compositor;

        // Create SpotLight and set its properties
        SpotLight spotLight = compositor.CreateSpotLight();
        spotLight.InnerConeAngleInDegrees = 50f;
        spotLight.InnerConeColor = Colors.FloralWhite;
        spotLight.OuterConeAngleInDegrees = 0f;
        spotLight.ConstantAttenuation = 1f;
        spotLight.LinearAttenuation = 0.253f;
        spotLight.QuadraticAttenuation = 0.58f;

        // Associate CompositionLight with XamlLight
        this.CompositionLight = spotLight;

        // Define resting position Animation
        Vector3 restingPosition = new Vector3(200, 200, 400);
        CubicBezierEasingFunction cbEasing = compositor.CreateCubicBezierEasingFunction( new Vector2(0.3f, 0.7f), new Vector2(0.9f, 0.5f));
        _offsetAnimation = compositor.CreateVector3KeyFrameAnimation();
        _offsetAnimation.InsertKeyFrame(1, restingPosition, cbEasing);
        _offsetAnimation.Duration = TimeSpan.FromSeconds(0.5f);

        spotLight.Offset = restingPosition;

        // Define expression animation that relates light's offset to pointer position 
        CompositionPropertySet hoverPosition = ElementCompositionPreview.GetPointerPositionPropertySet(targetElement);
        _lightPositionExpression = compositor.CreateExpressionAnimation("Vector3(hover.Position.X, hover.Position.Y, height)");
        _lightPositionExpression.SetReferenceParameter("hover", hoverPosition);
        _lightPositionExpression.SetScalarParameter("height", 15.0f);

        // Configure pointer entered/ exited events
        targetElement.PointerMoved += TargetElement_PointerMoved;
        targetElement.PointerExited += TargetElement_PointerExited;

        // Add UIElement to the Light's Targets
        HoverLight.AddTargetElement(GetId(), targetElement);
    }

    private void MoveToRestingPosition()
    {
        // Start animation on SpotLight's Offset 
        CompositionLight?.StartAnimation("Offset", _offsetAnimation);
    }

    private void TargetElement_PointerMoved(object sender, PointerRoutedEventArgs e)
    {
        if (CompositionLight == null) return;

        // touch input is still UI thread-bound as of the Creators Update
        if (e.Pointer.PointerDeviceType == Windows.Devices.Input.PointerDeviceType.Touch)
        {
            Vector2 offset = e.GetCurrentPoint((UIElement)sender).Position.ToVector2();
            (CompositionLight as SpotLight).Offset = new Vector3(offset.X, offset.Y, 15);
        }
        else
        {
            // Get the pointer's current position from the property and bind the SpotLight's X-Y Offset
            CompositionLight.StartAnimation("Offset", _lightPositionExpression);
        }
    }

    private void TargetElement_PointerExited(object sender, PointerRoutedEventArgs e)
    {
        // Move to resting state when pointer leaves targeted UIElement
        MoveToRestingPosition();
    }

    protected override void OnDisconnected(UIElement oldElement)
    {
        // Dispose Light and Composition resources when it is removed from the tree
        HoverLight.RemoveTargetElement(GetId(), oldElement);
        CompositionLight.Dispose();
        _lightPositionExpression.Dispose();
        _offsetAnimation.Dispose();
    }

    protected override string GetId() => typeof(HoverLight).FullName;
}

Now, with the HoverLight class done, we can add both the AmbLight and the HoverLight to previous ImageEffectBrush (find ImageEffectBrush in the last post):


<Grid>
            <Grid.Background>
                <brushes:ImageEffectBrush ImageUriString="ms-appx:///Images/Background.png" />
            </Grid.Background>
 
            <Grid.Lights>
                <lights:HoverLight/>
                <lights:AmbLight/>
            </Grid.Lights>
</Grid>

Note: To add a XamlLight in markup, your Min SDK version must be set to Creators Update, otherwise you can set it in the code behind.

For more information, go here to read more about using XamlLight and here to see the Lighting documentation.

Using CompositionPropertySets

When you want to use the values of the ScrollViewer’s Offset or the Pointer’s X and Y position (e.g. mouse cursor) to do things like animate effects, you can use ElementCompositionPreview to retrieve their PropertySets. This allows you to create amazingly smooth, 60 FPS animations that are not tied to the UI thread. These methods let you get the values from user interaction for things like animations and lighting.

Using ScrollViewerManipulationPropertySet

This PropertySet is useful for animating things like Parallax, Translation and Opacity.


// Gets the manipulation <ScrollViewer x:Name="MyScrollViewer"/>
CompositionPropertySet scrollViewerManipulationPropertySet = 
    ElementCompositionPreview.GetScrollViewerManipulationPropertySet(MyScrollViewer);

To see an example, go to the Smooth Interaction and Motion blog post in this series. There is a section devoted to using the ScrollViewerManipulationPropertySet to drive the animation.

Using PointerPositionPropertySet (new!)

New in the Creators Update, is the PointerPositionPropertySet. This PropertySet is useful for creating animations for lighting and tilt. Like ScrollViewerManipulationPropertySet, PointerPositionPropertySet enables fast, smooth and UI thread independent animations.

A great example of this is the animation mechanism behind Fluent Design’s RevealBrush, where you see lighting effects on the edges on the UIElements. This effect is created by a CompositionLight, which has an Offset property animated by an ExpressionAnimation using the values obtained from the PointerPositionPropertySet.


// Useful for creating an ExpressionAnimation
CompositionPropertySet pointerPositionPropertySet = ElementCompositionPreview.GetPointerPositionPropertySet(targetElement);
ExpressionAnimation expressionAnimation = compositor.CreateExpressionAnimation("Vector3(param.Position.X, param.Position.Y, height)");
expressionAnimation.SetReferenceParameter("param", pointerPositionPropertySet);

To get a better understanding of how you can use this to power animations in your app, let’s explore XamlLights and create a demo that uses the PointerPositionPropertySet to animate a SpotLight.

Enabling Translation Property – Animating a XAML Element’s Offset using Composition Animations

As discussed in our previous blog post, property sharing between the Framework Layer and the Visual Layer used to be tricky prior to the Creators Update. The following Visual properties are shared between UIElements and their backing Visuals:

  • Offset
  • Scale
  • Opacity
  • TransformMatrix
  • InsetClip
  • CompositeMode

Prior to the Creators update, Scale and Offset were especially tricky because, as mentioned before, a UIElement isn’t aware of changes to the property values on the hand-out Visual, even though the hand-out Visual is aware of changes to the UIElement. Consequently, if you change the value of the hand-out Visual’s Offset or Size property and the UIElement’s position changes due to a page resize, the UIElement’s previous position values will stomp all over your hand-out Visual’s values.

Now with the Creators Update, this has become much easier to deal with as you can prevent Scale and Offset stomping by enabling the new Translation property on your element, by way of the ElementCompositionPreview object.


ElementCompositionPreview.SetIsTranslationEnabled(Rectangle1, true);

//Now initialize the value of Translation in the PropertySet to zero for first use to avoid timing issues. This ensures that the property is ready for use immediately.

var rect1VisualPropertySet = ElementCompositionPreview.GetElementVisual(Rectangle1).Properties;
rect1VisualPropertySet.InsertVector3("Translation", Vector3.Zero);

Then, animate the visual’s Translation property where previously you would have animated its Offset property.


// Old way, subject to property stomping:
visual.StartAnimation("Offset.Y", animation);
// New way, available in the Creators Update
visual.StartAnimation("Translation.Y", animation);

By animating a different property from the one affected during layout passes, you avoid any unwanted offset stomping coming from the XAML layer.

Wrapping up

In the past couple posts, we explored some of the new features of XAML and Composition Interop and how using Composition features in your XAML markup is easier than ever. From painting your UIElements with CompositionBrushes and applying lighting, to smooth off-UIThread animations, the power of the Composition API is more accessible than ever.

In the next post, we’ll dive deeper into how you can chain Composition effects to create amazing materials and help drive the evolution of Fluent Design.

Resources

Working with Brushes and Content – XAML and Visual Layer Interop, Part One

The Composition APIs empower Universal Windows Platform (UWP) developers to do beautiful and powerful things when they access the Visual Layer. In the Windows 10 Creators Update, we made working with the Visual Layer much easier with new, powerful APIs.

In this blog series, we’ll cover some of these improvements in the Creators Update and take a look at the following APIs:

  • In Part 1, today’s post:
    • XamlCompositionBrushBase – easily paint a XAML UIElement with a CompositionBrush
    • LoadedImageSurface – load an image easily and use with Composition APIs
  • In Part 2, we’ll look at:
    • XamlLights – apply lights to your XAML UI with a single line of XAML
    • PointerPositionPropertySet – create 60 FPS animations using pointer position, off the UI thread!
    • Enabling the Translation property – animate a XAML UI Element using Composition animation

If you’d like to review the previously available ElementCompositionPreview APIs, for example working with “hand-in” and “hand-out” Visuals, you can quickly catch up here.

Using XamlCompositionBrushBase

One of the benefits of the new Composition and XAML interop APIs is the ability to use a CompositionBrush to directly paint a XAML UIElement rather than being limited to XAML brushes only. For example, you can create a CompositionEffectBrush that applies a tinted blur to the content beneath and use the brush to paint a XAML rectangle that can be included in the XAML markup

This is accomplished by using the new abstract class XamlCompositionBrushBase available in the Creators Update. To use it, you subclass XamlCompositionBrushBase to create your own XAML Brush that can be used in your markup. As seen the example code below, the XamlCompositionBrushBase exposes a CompositionBrush property that you set with your effect (or any CompositionBrush) and it will be applied to the XAML element.

This effectively replaces the need to manually create SpriteVisuals with SetElementChild for most effect scenarios. In addition to needing less code to create and add an effect to the UI, using a Brush means you get the following added benefits for free:

  • Theming and Styling
  • Binding
  • Resource and Lifetime management
  • Layout aware
  • PointerEvents
  • HitTesting and other XAML-based advantages

Microsoft, as part of the Fluent Design System, has included a few Brushes in the Creators Update that leverage the features of XamlCompositionBrushBase:

Building a Custom Composition Brush

Let’s create a XamlCompositionBrush of our own to see how simple this can be.  Here’s what we’ll create:

To start, let’s create a very simple Brush that applies an InvertEffect to content under it. First, we’ll need to make a public sealed class that inherits from XamlCompositionBrushBase and override two methods:

  • OnConnected
  • OnDisconnected

Let’s dive into the code. First, create your Brush class, which inherits from XamlCompositionBrushBase:


public class InvertBrush : XamlCompositionBrushBase
{
    protected override void OnConnected()
    {
        if (CompositionBrush == null)
        {
            // 1 - Get the BackdropBrush, this gets what is behind the UI element
            var backdrop = Window.Current.Compositor.CreateBackdropBrush();

            // CompositionCapabilities: Are effects supported? If not, return.
            if (!CompositionCapabilities.GetForCurrentView().AreEffectsSupported())
            { 
               return;
            }
                
            // 2 - Create your Effect
            // New-up a Win2D InvertEffect and use the BackdropBrush as its Source
            // Note – To use InvertEffect, you'll need to add the Win2D NuGet package to your project (search NuGet for "Win2D.uwp")
            var invertEffect = new InvertEffect
            {
                Source = new CompositionEffectSourceParameter("backdrop")
            };

            // 3 - Set up the EffectFactory
            var effectFactory = Window.Current.Compositor.CreateEffectFactory(invertEffect);

            // 4 - Finally, instantiate the CompositionEffectBrush
            var effectBrush = effectFactory.CreateBrush();

            // and set the backdrop as the original source 
            effectBrush.SetSourceParameter("backdrop", backdrop);

            // 5 - Finally, assign your CompositionEffectBrush to the XCBB's CompositionBrush property
            CompositionBrush = effectBrush;
         }
    }

    protected override void OnDisconnected()
    {
        // Clean up
        CompositionBrush?.Dispose();
        CompositionBrush = null;
    }
}

There are a few things to call out in the code above.

  • In the OnConnected method, we get a CompositionBackdropBrush. This allows you to easily get the pixels behind the UIElement.
  • We use fallback protection. If the user’s device doesn’t have support for the effect(s), then just return.
  • Next, we create the InvertEffect and use the backdropBrush for the Effect’s Source.
  • Then, we pass the finished InvertEffect to the CompositionEffectFactory.
  • Finally, we get an EffectBrush from the factory and set the XamlCompositionBrushBase.CompositionBrush property with our newly created effectBrush.

Now you can use it in your XAML. For example, let’s apply it to a Grid on top of another Grid with a background image:


<Grid>
    <Grid.Background>
        <ImageBrush ImageSource="ms-appx:///Images/Background.png"/>
    </Grid.Background>
    <Grid Width="300" Height="300" 
               HorizontalAlignment="Center"
               VerticalAlignment="Center">
        <Grid.Background>
            <brushes:InvertBrush />
        </Grid.Background>
    </Grid>
</Grid>

Now that you know the basics of creating a brush, let’s build an animated effect brush next.

Creating a Brush with Animating Effects

Now that you see how simple it is to create a CompositionBrush, let’s create a brush that applies a TemeratureAndTint effect to an image and animate the Temperature value:

We start the same way we did with the simple InvertBrush, but this time we’ll add a DependencyProperty,  ImageUriString, so that we can load an image using LoadedImageSurface in the OnConnected method.


public sealed class ImageEffectBrush : XamlCompositionBrushBase
    {
        private LoadedImageSurface _surface;
        private CompositionSurfaceBrush _surfaceBrush;
 
        public static readonly DependencyProperty ImageUriStringProperty = DependencyProperty.Register(
            "ImageUri",
            typeof(string),
            typeof(ImageEffectBrush),
            new PropertyMetadata(string.Empty, OnImageUriStringChanged)
        );
 
        public string ImageUriString
        {
            get => (String)GetValue(ImageUriStringProperty);
            set => SetValue(ImageUriStringProperty, value);
        }
 
        private static void OnImageUriStringChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
        {
            var brush = (ImageEffectBrush)d;
            // Unbox and update surface if CompositionBrush exists     
            if (brush._surfaceBrush != null)
            {
                var newSurface = LoadedImageSurface.StartLoadFromUri(new Uri((String)e.NewValue));
                brush._surface = newSurface;
                brush._surfaceBrush.Surface = newSurface;
            }
        }
 
        protected override void OnConnected()
        {
            // return if Uri String is null or empty
            if (string.IsNullOrEmpty(ImageUriString))
                return;
 
            // Get a reference to the Compositor
            Compositor compositor = Window.Current.Compositor;
 
            // Use LoadedImageSurface API to get ICompositionSurface from image uri provided
            _surface = LoadedImageSurface.StartLoadFromUri(new Uri(ImageUriString));
 
            // Load Surface onto SurfaceBrush
            _surfaceBrush = compositor.CreateSurfaceBrush(_surface);
            _surfaceBrush.Stretch = CompositionStretch.UniformToFill;
 
            // CompositionCapabilities: Are Tint+Temperature and Saturation supported?
            bool usingFallback = !CompositionCapabilities.GetForCurrentView().AreEffectsSupported();
            if (usingFallback)
            {
                // If Effects are not supported, Fallback to image without effects
                CompositionBrush = _surfaceBrush;
                return;
            }
 
            // Define Effect graph (add the Win2D.uwp NuGet package to get this effect)
            IGraphicsEffect graphicsEffect = new SaturationEffect
            {
                Name = "Saturation",
                Saturation = 0.3f,
                Source = new TemperatureAndTintEffect
                {
                    Name = "TempAndTint",
                    Temperature = 0,
                    Source = new CompositionEffectSourceParameter("Surface"),
                }
            };
 
            // Create EffectFactory and EffectBrush 
            CompositionEffectFactory effectFactory = compositor.CreateEffectFactory(graphicsEffect, new[] { "TempAndTint.Temperature" });
            CompositionEffectBrush effectBrush = effectFactory.CreateBrush();
            effectBrush.SetSourceParameter("Surface", _surfaceBrush);
 
            // Set EffectBrush to paint Xaml UIElement
            CompositionBrush = effectBrush;
 
            // Trivial looping animation to demonstrate animated effect
            ScalarKeyFrameAnimation tempAnim = compositor.CreateScalarKeyFrameAnimation();
            tempAnim.InsertKeyFrame(0, 0);
            tempAnim.InsertKeyFrame(0.5f, 1f);
            tempAnim.InsertKeyFrame(1, 0);
            tempAnim.Duration = TimeSpan.FromSeconds(5);
            tempAnim.IterationBehavior = AnimationIterationBehavior.Count;
      tempAnim.IterationCount = 10;
             effectBrush.Properties.StartAnimation("TempAndTint.Temperature", tempAnim);
        }
 
        protected override void OnDisconnected()
        {
            // Dispose Surface and CompositionBrushes if XamlCompBrushBase is removed from tree
            _surface?.Dispose();
            _surface = null;
 
            CompositionBrush?.Dispose();
            CompositionBrush = null;
        }
    }

There are some new things here to call out that are different from the InvertBrush:

  • We use the new LoadedImageSurface API to easily load an image in the OnConnected method, but also when the ImageUriString value changes. Prior to Creators Update, this required a hand-in Visual (a SpriteVisual, painted with an EffectBrush, which was handed back into the XAML Visual Tree). See the LoadedImageSurface section later in this article for more details.
  • Notice that we gave the effects a Name value. In particular, TemperatureAndTintEffect, uses the name “TempAndTint.” This is required to animate properties as it is used for the reference to the effect in the AnimatableProperties array that is passed to the effect factory. Otherwise, you’ll encounter a “Malformed animated property name” error.
  • After we assign the CompositionBrush property, we created a simple looping animation to oscillate the value of the TempAndTint from 0 to 1 and back every 5 seconds.

Let’s take a look at an instance of this Brush in markup:


<Grid>
            <Grid.Background>
                <brushes:ImageEffectBrush ImageUriString="ms-appx:///Images/Background.png"/>
            </Grid.Background>
</Grid>

For more information on using XamlCompositionBrushBase, see here. Now, let’s take a closer look at how easy it is now to bring in images to the Visual layer using LoadedImageSurface

Loading images with LoadedImageSurface

With the new LoadedImageSurface class, it’s never been easier to load an image and work with it in the visual layer. The class has the same codec support that Windows 10 has via the Windows Imaging Component (see full list here), thus it supports the following image file types:

  • Joint Photographic Experts Group (JPEG)
  • Portable Network Graphics (PNG)
  • Bitmap (BMP)
  • Graphics Interchange Format (GIF)
  • Tagged Image File Format (TIFF)
  • JPEG XR
  • Icons (ICO)

NOTE: When using an animated GIF, only the first frame will be used for the Visual, as animation is not supported in this scenario.

To load in an image, you can use one of the four factory methods:

As you can see there are two ways to load an image: with a Uri or a Stream. Additionally, you have an option to use an overload to set the size of the image (if you don’t pass in a Size, it will decode to the natural size).


CompositionSurfaceBrush imageBrush = compositor.CreateSurfaceBrush();
LoadedImageSurface loadedSurface = LoadedImageSurface.StartLoadFromUri(new Uri("ms-appx:///Images/Photo.jpg"), new Size(200.0, 200.0));
imageBrush.Surface = loadedSurface;

This is very helpful when you need to load an image that will be used for your CompositionBrush (e.g. CompositionEffectBrush) or SceneLightingEffect (e.g. NormalMap for textures) as you no longer need to manually create a hand-in Visual (a SpriteVisual painted with an EffectBrush). In an upcoming post in this series, we will explore this further using NormalMap images with to create advanced lighting to create unique and compelling materials.

Using LoadedImageSurface with a Composition Island

LoadedImageSurface is also useful when loading an image onto a SpriteVisual inserted in XAML UI using ElementCompositionPreview. For this scenario, you can use the Loaded event to adjust the visual’s properties after the image has finished loading.

Here is an example of using LoadedImageSurface for a CompositionSurfaceBrush, then updating the SpriteVisual’s size with the image’s DecodedSize when the image is loaded:


private SpriteVisual spriteVisual;
private void LoadImage(Uri imageUri)
{
    CompositionSurfaceBrush surfaceBrush = Window.Current.Compositor.CreateSurfaceBrush();

    // You can load an image directly and set a SurfaceBrush's Surface property with it
    var loadedImageSurface = LoadedImageSurface.StartLoadFromUri(imageUri);
    loadedImageSurface.LoadCompleted += Load_Completed;
    surfaceBrush.Surface = loadedImageSurface;

    // We'll use a SpriteVisual for the hand-in visual
    spriteVisual = Window.Current.Compositor.CreateSpriteVisual();
    spriteVisual.Brush = surfaceBrush;

    ElementCompositionPreview.SetElementChildVisual(MyCanvas, spriteVisual);
}

private void Load_Completed(LoadedImageSurface sender, LoadedImageSourceLoadCompletedEventArgs args)
{
    if (args.Status == LoadedImageSourceLoadStatus.Success)
    {
        Size decodedSize = sender.DecodedSize;
        spriteVisual.Size = new Vector2((float)decodedSize.Width, (float)decodedSize.Height);
    }
}

There are some things you should be aware before getting started with LoadedImageSurface. This class makes working with images a lot easier, however you should understand the lifecycle and when images get decoded/sized. We recommend that you take a couple minutes and read the documentation before getting started.

Wrapping up

Using Composition features in your XAML markup is easier than ever. From painting your UIElements with CompositionBrushes and applying lighting, to smooth off-UIThread animations, the power of the Composition API is more accessible than ever.

In the next post, we’ll explore more new APIs like the new Translation property, using XamlLights in your XAML markup and how to create a custom light using the new PointerPositionPropertySet.

Resources

Smooth Interaction and Motion with the Visual Layer in Windows 10 Creators Update

The Composition APIs come with a robust animation engine that provides quick and fluid motion running in a separate process from your Universal Windows Platform (UWP) app. This ensures a consistent 60 frames per second when running your app on an IoT device as well as on a screaming gaming machine. It is, quite simply, fast. This is an essential capability for implementing the Fluent Design System which calls on us to create a sense of cinematic motion in our UWP apps.

The Composition APIs also provide something you probably have never had access to before: the ability to create high-performing, low-level manipulation-driven custom animations like the one shown above.  In the same way that we that want our visuals to be fast and smooth, we want our touch interactions to be sticky and responsive. Moving a visual with a finger or a digital pen should result in the visual element clinging to us no matter how fast we push and pull it across the display.

Even if a motion looks good, it also needs to feel good under the finger. It needs to maintain the illusion that we are interacting with a real object. It ought to possess the proper physics so that when we drag a visual across the screen and let go, it continues with the proper inertial movement. Similarly, user controls should provide the right amount of resistance when we pull and release them.

A fast and fluid animation system

The Visual Layer supports both keyframe animations as well as expression animations. If you have worked with XAML animations before, then you are probably already familiar with how keyframes work. In a keyframe animation, you set values for some property you want to change over time and also assign the duration for the change: in the example below, a start value, a middle value and then an ending value. The animation system will take care of tweening your animation – in other words, generating all the values between the ones you have explicitly specified based on the easing function you select. Whether Linear, or a Cubic Bezier, the animation system will use that to determine the values when interpolating.


CubicBezierEasingFunction cubicBezier = _compositor.CreateCubicBezierEasingFunction(new Vector2(.17f, .67f), new Vector2(1f, 1f));
ScalarKeyFrameAnimation blurAnimation = _compositor.CreateScalarKeyFrameAnimation();
blurAnimation.InsertKeyFrame(0.0f, 0.0f);
blurAnimation.InsertKeyFrame(0.5f, 100.0f);
blurAnimation.InsertKeyFrame(1.0f, 0.0f);
blurAnimation.Duration = TimeSpan.FromSeconds(4);
blurAnimation.IterationBehavior = AnimationIterationBehavior.Forever;
_brush.StartAnimation("Blur.BlurAmount", blurAnimation);

A keyframe animation is a fire-and-forget mechanism that is time based. There are situations, however, when you need your animations to be coordinated and driving each other instead of simply moving in synchronized fashion.

In the animation above (source code), each gray gear is animated based on the animation of the gear preceding it. If the preceding gear suddenly goes faster or reverses direction, it forces the following gear to do the same. Keyframe animations can’t create motion effects that work in this way, but expression animations can. They are able to do so because, while keyframe animations are time based, expression animations are reference based.

The critical code that hooks up the gears for animation is found in the following code sample, which uses the new Expression Builder Library—an open source component released alongside of the Creators Update to construct expression animations. The expression below says that the animation should reference and be driven by the RotationAngleInDegrees property of the Visual that is indicated by the parameter “previousGear”. In the next line, the current Visual’s RotationAngleInDegrees property is finally animated based on the value referred to in an expression.


private void ConfigureGearAnimation(Visual currentGear, Visual previousGear)
{
    // If rotation expression is null then create an expression of a gear rotating the opposite direction

    var _rotateExpression = previousGear.GetReference().RotationAngleInDegrees;

    // Start the animation based on the Rotation Angle in Degrees.
    currentGear.StartAnimation("RotationAngleInDegrees", _rotateExpression);
}

But if an animation can be driven by another animation, you may be wondering, couldn’t we also drive an animation with something more concrete like user input? Why, yes. Yes, we can.

The beauty of the ScrollViewer ManipulationPropertySet

Driving an animation from a ScrollViewer using XAML-Composition interop is fairly easy. With just a few lines of code, you can enhance the visuals of a pre-existing ScrollViewer control with a CompositionAnimation by taking advantage of the GetScrollViewerManipulationPropertySet method on the ElementCompositionPreview class. Using an animation expression, you can tie your animation to the Position of your ScrollViewer component.

You would use this technique if you wanted to add a parallax effect to your XAML or to create a sticky header that stays in place as content scrolls beneath it. In the demo illustrated below (source code), a ScrollViewer is even used to drive a parallax effect on a ListView.

Adding parallax behavior to a XAML page can be accomplished in just a few lines.


// Note: We're not using the ScrollViewer's offset values directly. Instead, we use this PropertySet which holds the position values of the ScrollViewer in real-time.
var scrollPropSet = _scrollProperties.GetSpecializedReference<ManipulationPropertySetReferenceNode>();
var startOffset = ExpressionValues.Constant.CreateConstantScalar("startOffset", 0.0f);
var parallaxValue = 0.5f;
var itemHeight = 0.0f;
var parallax = (scrollPropSet.Translation.Y + startOffset - (0.5f * itemHeight));
_parallaxExpression = parallax * parallaxValue - parallax;
_parallaxExpression.SetScalarParameter("StartOffset", (float)args.ItemIndex * visual.Size.Y / 4.0f);
visual.StartAnimation("Offset.Y", _parallaxExpression);

The even more beautiful InteractionTracker

Driving expression animations with a ScrollViewer is extremely powerful, but what if you want to drive animations using touch gestures that aren’t limited to a pan/zoom gesture? Additionally, when using the ScrollViewer’s manipulations, your animations are linked to the UI thread responsiveness and can lose that buttery-smooth feel when the UI thread gets bogged down.

What if you want to pull items toward you with your finger, as in the demo below (source code), or animate multiple flying images across and into the screen as happens in the demo at the top of this post (source code)?

In order to achieve these effects, you would use the new InteractionTracker and VisualInteractionSource classes. InteractionTracker is a state machine that can be driven by active input. InteractionTracker also maintains a series of properties like Position and ScalePosition as part of maintaining the state. This is what you hook up to your animations. The VisualInteractionSource class, on the other hand, determines what kind of input you will use to drive your InteractionTracker and also when to start handling input (touch in particular).

The following sample code demonstrates a basic implementation of an InteractionTracker. The viewportVisual is simply the backing Visual for the root element on the page. You use this as the VisualInteractionSource for the tracker. In doing so, you specify that you are tracking X and Y manipulations. You also indicate that you want to track inertial movement.


_tracker = InteractionTracker.Create(_compositor);

var interactionSource = VisualInteractionSource.Create(viewportVisual);

interactionSource.PositionXSourceMode = InteractionSourceMode.EnabledWithInertia;
interactionSource.PositionYSourceMode = InteractionSourceMode.EnabledWithInertia;

_tracker.InteractionSources.Add(interactionSource);

Hooking the tracker up to an expression animation works basically the same way as hooking up a gear Visual to another gear Visual, as you did earlier. You call the CreateExpressionAnimation factory method on the current Compositor and reference the Position property of the tracker.


ar positionExpression = _compositor.CreateExpressionAnimation("-tracker.Position");
positionExpression.SetReferenceParameter("tracker", _tracker);

contentVisual.StartAnimation("Offset", positionExpression);

This code uses the InteractionTracker’s position to produce a smooth animation for the Offset of the Visual. You can also power your Blur and Opacity animations for your other Visuals as well. This will have a result where all three animations work together, with values based on how far the user dragged their finger, to result in an amazingly fluid visual experience. Run the demo and try it for yourself (source code).

Those are the basics of driving any animation from any input. What you do with this amazing new power is entirely up to you.

Wrapping up

Expression animations and Visual Layer Interactions are both topics that can become very deep very fast. To help you through these deeper waters, we highly recommend the following videos and articles:

Smooth as Butter Animations in the Visual Layer with the Windows 10 Creators Update

The Windows 10 Creators Update marks the third major release of the Windows UI platform APIs. With each release, an attempt is frequently made to simplify features introduced in prior releases. This encourages Universal Windows Platform (UWP) developers to standardize on these features. The new hide and show implicit animations are an example of this.

At the same time, too much standardization can potentially lead to conformity, so with each new release more powerful visual features like the new custom animations are also added, which allow developers who are willing and able to dive into them to customize their user interfaces and stand out from the crowd. This inherent tension between ease of use and the power to customize rewards developers for their efforts while also making sure that no one gets left behind.

Hide and show animations for page transitions

Page transitions, often accompanied by state transitions as visual elements, are added to the visual tree of the new page. In fact, a lot of interactivity in UWP simply involves deciding which content to show and which content to hide as the state of the app changes. More often than not, this is tied to changing the value of the Visibility properties of the elements on the screen.

In the Creators Update, two new implicit animation techniques have been added to help you make these transitions more fluid: ElementCompositionPreview.SetImplicitShowAnimation and ElementCompositionPreview.SetImplicitHideAnimation. Whenever a UIElement is loaded or when that element’s Visibility property is set to Visible, the implicit animation associated with it using SetImplicitShowAnimation will play. Similarly, whenever the user navigates away from a page or when a UIElement is hidden, an animation associated with it using the SetImplicitHideAnimation method will be invoked. These two mechanisms make it easier for you to include motion as an inherent aspect of all your visual elements, while providing a seamless experience for your users.

Connected animations

Implicit animations are great for animating controls inside a page. For navigation transitions between pages, however, the Visual Layer provides a different mechanism known as connected animations to help you make your UI even sweeter. Connected animations help the user stay oriented when she is performing common tasks such as context switching from a list of items to a details page.

The Windows UI platform APIs provide a class named the ConnectedAnimationService to coordinate animations between the source page and the destination page during navigation. You access the service by calling the static GetForCurrentView method. Then in the source page, you invoke PrepareToAnimate, passing in a unique key and the image that should be used for the transition animation.


ConnectedAnimationService.GetForCurrentView().PrepareToAnimate("MyUniqueId", image);

In the destination page, you retrieve the image from your ConnectedAnimationService service and invoke TryStart on the ConnectedAnimation while passing in the destination UIElement.


var animation = ConnectedAnimationService.GetForCurrentView().GetAnimation("MyUniqueId");
if (animation != null)
{
    animation.TryStart(DestinationImage);
};

In the Anniversary Update you did not have much control over this animation technique. Everyone got pretty much the same standard one. With the Creators Update, on the other hand, you have lots of new superpowers to personalize your transitions with:

  • Coordinated animations
  • Custom animations
  • Better image animations

Just to reiterate the point made in the introduction, the goal in designing the Windows UI platform APIs is to provide an awesome experience out of the box so you can copy the standard samples and get beautiful, fast and visually appealing visuals. At the same time, this shouldn’t ever take away from your ability to personalize the user experience to create something truly unique and wonderful with powerful new tools, like coordinated animations and custom animations.

Coordinated animations

A coordinated animation is a type of animation that appears alongside your connected animation and which works in coordination with your connected animation target. A coordinated animation gives extra visual flair to your page transition.

In the coordinated animation sample above, caption text that is not present in the source page is added to the destination page. The caption text is animated in tandem with the connected animation. We are doing two things here (in designer terms): providing context between the source and the destination using our connected animation while also adding visual interest with a coordinated animation at the destination. In user experience terms, though, all we’re doing is making the app’s transition animations look really cool.

Coordinated animations are fortunately also easy to implement. The TryStart method of the ConnectedAnimation class provides an override that allows you to pop in an array of visual elements you want to animate in a coordinated fashion. Let’s say that your caption text is in a visual element that you’ve named “DescriptionRoot.” You can add this as a coordinated animation by tweaking the previous code like so:


var animation = ConnectedAnimationService.GetForCurrentView().GetAnimation("MyUniqueId");
if (animation != null)
{
    animation.TryStart(DestinationImage, new UIElement[] { DescriptionRoot });
};

That’s a lot of power packed into a little argument.

Custom animations

By default, the connected animations in the navigation sample move in a straight line from the origin position in the source page to the target position in the destination page. If you select a box in the far-left column, it will move more or less straight up, while if you select a box in the top row, it will more or less move directly left to get to that target position. But what if you could put some English on this?

You can with custom animations, introduced in the Creators Update. The custom animations feature lets you modulate your transitions in four ways:

  • Crossfade – Lets you customize how elements crossfade as source element reaches destination
  • OffsetX – Lets you customize the X channel of Offset
  • OffsetY – Lets you customize the Y channel of Offset
  • Scale – Lets you customize scale of the element as it animates

In order to customize a particular part of a connected animation, you will need to create a keyframe animation and add it to your page transition using the SetAnimationComponent call like so:


var animation = ConnectedAnimationService.GetForCurrentView().GetAnimation("MyUniqueId");

var customXAnimation = Window.Compositor.CreateScalarKeyFrameAnimation();
customXAnimation.Duration = ConnectedAnimationService.GetForCurrentView().DefaultDuration;
customXAnimation.InsertExpressionKeyFrame(0.0f, "StartingValue");
customXAnimation.InsertExpressionKeyFrame(0.5f, "FinalValue + 25");
customXAnimation.InsertExpressionKeyFrame(1.0f, "FinalValue");

animation.SetAnimationComponent(ConnectedAnimationComponent.OffsetX, customXAnimation);

Note that you use expressions to get the starting and ending values of the connected animation.

Awesome image animations

The Creators Update also introduces improved image interpolation for connected animations where the image size and even the relative dimensions are changing between the source and the destination—for instance transitioning from a square to a rectangular image.

This interpolation happens automagically so you have less to worry about.

Implicit animation support for property sets and shadows

Finally, animation capabilities are also extended in the Creators Update by allowing you to apply implicit animations to property sets and shadows.

This change provides developers with even more creative flexibility and the ability to modify shadows in interesting new ways, as shown in the code sample below.


var shadowBlurAnimation = compositor.CreateScalarKeyFrameAnimation();
shadowBlurAnimation.InsertExpressionKeyFrame(1.0f, "this.FinalValue");
shadowBlurAnimation.Duration = TimeSpan.FromSeconds(1);
shadowBlurAnimation.Target = "BlurRadius";

//Associating animations with triggers 
implicitAnimationShadow["BlurRadius"] = shadowBlurAnimation;
implicitAnimationShadow["Opacity"] = shadowOpacityAnimation;
implicitAnimationShadow["Scale"] = shadowScaleAnimation;

implicitAnimationVisual["Translation"] = translationAnimation;
            

//Applying Implicit Animations to objects 
content.Properties.ImplicitAnimations = implicitAnimationVisual;
shadow.DropShadow.ImplicitAnimations = implicitAnimationShadow;

Wrapping up

The visual power being made available to developers through the Windows UI platform APIs have basically always been a part of the UI Framework. They just haven’t always been accessible until now. Think of this as a UI nuclear reactor being handed over to you to play with. With this awesome power, however, also comes the responsibility to create sweet UI and beautiful interactions. Go forth and be amazing.

To learn more about the topics covered in this post, you are encouraged to voraciously consume the following articles and videos:

Interop between XAML and the Visual Layer

The Composition APIs empower Universal Windows Platform (UWP) developers to do beautiful and powerful things when they drop down to the Visual Layer (as in the sample below). One of the remarkable things about these APIs is that they are designed, end-to-end, to integrate with the XAML Layer. Because of this built-in interoperability, the Composition APIs allow you to visually enhance your current XAML code without needing to replace it.

IMAGE1

This post will cover the specific APIs for pulling Visuals out of your XAML as well as pushing Visuals back into your layout. These are also known as hand-out Visuals and hand-in Visuals. Additionally, we’ll cover some details of property sharing between the Framework Layer and the Visual Layer which you will find helpful to get under your belt. We’ll finish off with a dessert recipe (hint, it includes frosting) that demonstrates how the interop between XAML and the Visual Layer is used to create sweet effects.

Let’s talk about hand-out Visuals

Each XAML UIElement in your page layout has a backing Visual that is used for rendering. Because Visuals can contain other Visuals, the tree made up of Visuals tends to be larger than the logical tree made up of XAML elements. If you have been working with XAML for a while, you are probably already familiar with this concept of visual trees and logical trees.

IMAGE2

Using the interop class ElementCompositionPreview, which is the workhorse of XAML-Composition interop, you can grab the backing visual for any page element by calling the GetElementVisual method.


Visual backingVisual = ElementCompositionPreview.GetElementVisual(MyImage);

In fact, grabbing the root backing Visual for your page, rather than a specific element, is a common way to retrieve the current Compositor instance.


// Get the current compositor
Compositor compositor = ElementCompositionPreview.GetElementVisual(this).Compositor;

The Visual you receive from ElementCompositionPreview in this way is referred to as a hand-out Visual. Several properties of a hand-out Visual are effectively shared between your XAML and the Composition API so that changes to UIElement properties — such as those related to position, height and width — can alter the Offset and Size properties of the Visual. Some care should be taken when working with these properties, however. While hand-out Visuals receive property changes made to UIElements, UIElements will not read property changes made to hand-out Visuals. The flow of property information flows in only one direction.

Welcome to Composition Island

The SetElementChildVisual method of ElementCompositionPreview plays the role of equal and opposite reaction to the GetElementVisual method. It allows you to take a Visual you have created and add it to a UIElement’s visual tree. When a Visual is added in this way, it is called a hand-in Visual. It doesn’t participate in property sharing the way hand-out Visuals do. Instead, a hand-in Visual acts as a Composition Island: an isolated piece of Visual-based content surrounded by a sea of XAML.

IMAGE3

By default, a hand-in Visual is added as the last member of the UIElement’s visual tree, placing it prominently on top of other child Visuals.


Visual dataIsland = _compositor.CreateSpriteVisual();
ElementCompositionPreview.SetElementChildVisual(MyImage, dataIsland);

ElementCompositionPreview also has a third method, GetElementChildVisual, which takes a UIElement as an argument and returns the hand-in Visual you previously added to the element’s visual tree.

Things you should know about property sharing and property stomping

As discussed above, property sharing between the Framework Layer and the Visual Layer can be a little tricky if you aren’t careful. The following Visual properties are shared between UIElements and their backing Visuals:

  • Offset
  • Size
  • Opacity
  • TransformMatrix
  • Clip
  • CompositeMode

Size and Offset are especially tricky because, as mentioned before, a UIElement isn’t aware of changes to the property values on the hand-out Visual, even though the hand-out Visual is aware of changes to the UIElement. Consequently, if you change the value of the hand-out Visual’s Offset or Size property and the UIElement’s position changes due to a page resize, the UIElement’s previous position values will stomp all over your hand-out Visual’s values.

There is a simple trick that will allow you to avoid property stomping, though. Place your UIElement inside a Grid control or Border control. Then instead of setting position properties directly on your UIElement, you should set them on the wrapper instead.


<Border x:Name="ElementWrapper" Margin="5" VerticalAlignment="Top" Width="300">
    <Image x:Name="MyImage" Source="Assets/Lenna.png"/>
</Border> 

By doing this, the UIElement’s position appears to never change from its default settings, and so it doesn’t draw any attention during the layout pass, thus avoiding any unwanted property stomping. Additionally, changes to the hand-out Visual’s Offset property will be relative to the position of the wrapper.

You can find out more about property stomping in Robert Mikhayelyan’s blog post about Changes to XAML-Composition Interop.

Recipe: How to add frosting to the Visual Layer

In the following recipe, you will add a frosted glass effect to a XAML element that is overlaid on top of an image. This effect illustrates both the GetElementVisual and SetElementChildVisual methods at work. A more detailed example of this recipe can be found in the Dev Center post Visual Layer with XAML.

Let’s start with a simple image control.

IMAGE4

While you could add the frosting effect directly to the Image control, placing a canvas over it and then applying a Composition effect to the canvas better demonstrates how smoothly XAML and the Visual Layer work together.


<Border x:Name="ElementWrapper" Margin="5" VerticalAlignment="Top" Height="300" Width="300">
    <Image x:Name="MyImage" Source="Assets/Lenna.png"/>
</Border>
<Canvas x:Name="FrostHost" Width="130" Height="300" Margin="5"
    HorizontalAlignment="Right" VerticalAlignment="Top"/>

There are basically four steps involved in adding frosting to the FrostHost element:

  1. Grab the backing Visual for FrostHost with GetElementVisual
  2. Define and create the effect
  3. Create a new Visual and apply the effect to it
  4. Insert the new Visual into FrostHost’s visual tree with SetElementChildVisual

In the first step, you can kill two birds with one stone by grabbing the backing Visual with GetElementVisual and retrieving the current Compositor at the same time.


Visual hostVisual = ElementCompositionPreview.GetElementVisual(FrostHost);
Compositor compositor = hostVisual.Compositor;

You define the frost effect by creating a blur effect and chaining it to a tint effect by setting one of its sources to a solid color. The compositor is then used to actually create the effect.


// Create a frosty glass effect
var frostEffect = new GaussianBlurEffect
{
    BlurAmount = 15.0f,
    BorderMode = EffectBorderMode.Hard,
    Source = new ArithmeticCompositeEffect
    {
        MultiplyAmount = 0,
        Source1Amount = 0.5f,
        Source2Amount = 0.5f,
        Source1 = new CompositionEffectSourceParameter("backdropBrush"),
        Source2 = new ColorSourceEffect
        {
            Color = Color.FromArgb(255, 245, 245, 245)
        }
    }
};

// Create an instance of the effect and set its source to a CompositionBackdropBrush
var effectFactory = compositor.CreateEffectFactory(frostEffect);
var backdropBrush = compositor.CreateBackdropBrush();
var effectBrush = effectFactory.CreateBrush();

effectBrush.SetSourceParameter("backdropBrush", backdropBrush);

Next, you create a new SpriteVisual using your compositor instance and add the frosting effect you created above to it.


// Create a Visual to contain the frosted glass effect
var frostVisual = compositor.CreateSpriteVisual();
frostVisual.Brush = effectBrush;

All that remains is to insert the new Visual back into your XAML as a hand-in Visual by using SetElementChildVisual. As a final touch, you can also sync your hand-in Visual with your hand-out Visual.


// Add the blur as a child of the host in the visual tree
ElementCompositionPreview.SetElementChildVisual(FrostHost, frostVisual);

// Make sure size of frost host and frost visual always stay in sync
var bindSizeAnimation = compositor.CreateExpressionAnimation("hostVisual.Size");
bindSizeAnimation.SetReferenceParameter("hostVisual", hostVisual);

The result is a picture half-frosted over by the FrostHost canvas placed on top of it. What’s interesting here is that the frosting was not added to the picture but to a different element. As we would hope, though, the effect passes through to the underlying image without any artifacts or airspace issues.

IMAGE5

Wrapping up

The ability to pull hand-out Visuals from your XAML and to insert Composition Islands back into it provides a highly accessible way to go back and forth between the Framework Layer and the Visual Layer. Understanding the central role of the ElementCompositionPreview class is important to accomplishing smooth integration between your XAML and the Composition APIs. If you would like to go even deeper into the Interop story behind Windows.UI.Composition, we highly recommend the following resources.

Get Started with Visual Studio.