Tag Archives: creators update

See What’s New with Windows Ink in the Windows 10 Creators Update

Windows Ink is about transforming the way we think about computers, from a tool that is great at getting things done, to one that harnesses your personality and your emotions into the things you create. It’s about bringing back the human aspects that a mouse and keyboard (and even touch) cannot express fully, it’s about making personal computers more personal, and an extension of yourself, not just a tool. We want you to feel empowered to create from the moment you pick up the pen, and have the confidence that Windows understands you, knows what you want to do – by understanding your handwriting, your words and your expression. This is the journey we’re on.

With the Creators Update, Windows Ink is now better than ever! When used with the Surface Dial, it allows you to discover new ways to work and interact with Windows. With Windows Ink, we continue to make it possible for you to do more than with pen and paper. Applications like Photos and Maps have added incredible inking functionality in the last year, and continue to evolve and expand. With Paint 3D in the Creators Update, Windows Ink can now create 3D objects! As we evolve what Ink means to users, we’re also introducing new Smart Ink capabilities to Windows Ink. These capabilities allows developers to understand the ink that is being laid down by the user, using AI to help create, connect and complete user actions on ink. We’ve also improved and added features to the building blocks for Windows Ink, introducing new stencils and adding tilt support to create a richer drawing experience.

Devices that support the Pen on Windows have also doubled in the last year, and is on track to double again in the next year! We’re seeing high demand not just for devices, but also for applications that support ink. To make it easier to find compatible pens, Wacom has partnered with us to develop the Bamboo Ink Pen. This pen will be in market in summer and supports almost all Windows PCs that are pen-capable. It features the Microsoft Pen Protocol (MPP), which is based on Surface Pen technology. In addition, we are also excited that the Surface Dial is now available in more countries, like Australia, Canada and New Zealand, giving more people an opportunity to try this incredible new input device. In addition, new hardware from our OEM partners, like the Dell Canvas 27, are shipping soon and takes advantage of the same RadialController APIs that are used for the dial. As a developer building for the Surface Dial today, it means that you are ready for all the new hardware that our OEM partners will bring to the ecosystem.

The progress we’ve made with Windows Ink would not have been possible without the feedback and passion you developers bring to us. With over a thousand inking applications in the store and growing everyday, with well over half of the top 10 paid store apps being ink apps, there is incredible enthusiasm and interest in this space. This is an incredible opportunity that you have embraced with us, and it inspires us to do more in each Windows release.

What’s new with Windows Ink platform?

Ink is the ultimate way humans can express themselves, it opens up new opportunities for application developers to differentiate, and helps make their applications stand out. From the latest fads like adult coloring books to simple games like tic-tac-toe, to applications that help you organize your life, there is just so much opportunity to build the next big thing in the inking space. We also know that people who use Windows Ink are more satisfied with their experience, what they look for, and buy more inking applications. From the platform perspective, we have 2 ways that we help developers:

  • Make it as easy and quick for a developer to add inking into their application by providing controls that can be dropped in quickly into any application and get Windows Ink support.
  • Provide the most flexible platform building blocks for developers to innovate upon. This gives you the flexibility to choose where to start developing for Windows Ink.

Introducing Smart Ink

Let’s start with a new building block that developers have access to in the Creators Update. Introducing Ink Analysis, this is the first of our family of Smart Ink capabilities that we are bringing to the platform. Smart Ink brings AI technology to not just understand what you write, but also helps connect the dots to what you may want to do. With Ink Analysis, it starts simple, with recognizing shapes and making that square you drew more perfect, but it can also do much more, like understanding you wrote words in squares and making it into an org chart using understanding about your organization. Our goal is to understand user intent and empower developers to turn it into rich digital constructs, as well as to leverage understanding from all parts of the system. Ink Analysis allows any developer to understand the ink they capture, whether it is handwriting, shapes, phone numbers, stock symbols, lists, document structure and more.  This is the same technology we debuted in Sticky Notes in the Window 10 Anniversary Update, and now it’s available for you to use! We can’t wait to see what you can do with this technology.

Here is an example of how to use Ink Analysis to recognize shapes.  For this snippet, we’ll use DirectInk to handle rendering the ink strokes.  Start by initializing an InkAnalyzer and connecting it with InkPresenter:


private void Initialize()
{
    inkAnalyzer = new InkAnalyzer();
    inkCanvas.InkPresenter.StrokesCollected += InkPresenter_StrokesCollected;
    inkCanvas.InkPresenter.StrokesErased += InkPresenter_StrokesErased;
}

// Whenever the user draws a new stroke, you copy the stroke into Ink Analyzer’s stroke collection
private void InkPresenter_StrokesCollected(InkPresenter sender, InkStrokesCollectedEventArgs args)
{
    inkAnalyzer.AddDataForStrokes(args.Strokes);
}

// When a stroke is erased in InkCanvas, remove the same stroke from Ink Analyzer's collection.
private void InkPresenter_StrokesErased(InkPresenter sender, InkStrokesErasedEventArgs args)
{
    foreach (var stroke in args.Strokes)
    {
        inkAnalyzer.RemoveDataForStroke(stroke.Id);
    }
}

Next you want to feed strokes to the analyzer. Commonly this is done via explicit user action (e.g. the user clicks a button) or after the user has been idle for a while.


inkAnalyzer.AnalyzeAsync();

The result is a tree representation of the whole document with different kinds of nodes, such as paragraph, line, list, word, and drawing. If for instance you want to find all the shapes in the ink, you can with the code below:


IReadOnlyList<IInkAnalysisNode> drawings = inkAnalyzer.AnalysisRoot.FindNodes(InkAnalysisNodeKind.InkDrawing);
foreach (IInkAnalysisNode drawing in drawings)
{
    var shape = (InkAnalysisInkDrawing)drawing;
    switch (shape.DrawingKind)
    {
        case InkAnalysisDrawingKind.Circle:
            // The user drew a circle. You can replace it with a perfect circle that goes through shape.Points.
            break;
        case InkAnalysisDrawingKind.Rectangle:
            // The user drew a rectangle. 
            // You can find the full list of supported shapes here.
            break;
    }
} 

If you want to learn more about Ink Analysis, you can watch the BUILD 2017 recorded video Enable Natural Pen Interaction by Using Ink Analysis to Better Understand Users’ Ink, download the Ink Analysis sample on GitHub or check out the Ink Analysis API Reference.

An improved Ink Toolbar

In the Anniversary Update we created a customizable set of inking tools, Ink Toolbar and Ink Canvas, that any developer can add to their own application with only two lines of markup.


<InkCanvas x:Name=“myInkCanvas”/>
<InkToolbar TargetInkCanvas=“{x:Bind myInkCanvas}”/>

Many of Microsoft’s first party applications have incorporated the inking tools to create engaging user experiences. For example, Photos added a calligraphy pen and the ability to draw on any photo in the gallery. Maps added a feature that lets you measure the distance of a route drawn on the map. Edge browser added inking on webpages. It has never been easier to add Windows Ink to your applications.

In the Creators Update, we continue our commitment to improving these controls! If you already use them in your applications, these improvements will benefit you with no additional work!

In response to users, the Creators Update introduces a new stencil, the protractor. This new stencil makes it easy for you to draw circles and arcs of any size. When drawing an arc, the protractor displays a readout that tells you the precise angle of the arc. You can also resize the stencil with just a pinch/zoom gesture with your fingers.

We’ve also made the ruler stencil better! Like the protractor, it now provides an angle readout that shows the ruler’s angle with the horizontal line. The ruler also snaps to 0, 45 and 90 degrees for easy access to the most common angles being used by our users.

You asked for an improve stroke preview in the Ink Toolbar, and in the Creators Update, we have it! We’re also make changes in the Ink Toolbar to work better with High Contrast themes, by automatically showing only colors that meet visibility requirements for the current user profile.

New Exciting Inking Capabilities


Today we announced the new Surface Pro and the new Surface Pen. Together they enable the next generation of inking capabilities that truly make writing digitally as natural as pen on paper. Here are some of the highlights:

  • Low latency Ink that virtually eliminates lag when you write
  • Tilt support to capture an additional dimension in digital inking
  • Ink that captures the entire spectrum of your expression with 4,096 levels of pressure sensitivity
  • Effortless inking with half the activation force required to being inking

Our customers have asked us for these capabilities, and they are finally here! From a developer perspective, if you already use the Windows Ink platform, all these capabilities show up in your application automatically! There are no changes required, and you are ready for the new Surface Pro, with the new Surface Pen.

Low latency Inking is a unique addition to Windows Ink. It is the result of a close partnership between hardware and software. The Pixelsense Accelerator chip in the new Surface Pro, is the first device to run Windows Ink acceleration code natively on hardware. This is how we achieve a new milestone in inking, virtually eliminating lag between the pen tip and the ink that flows out of it, creating the most natural writing experience with Windows Ink.

Tilt is another great addition to the Inking experience. The great news is, in addition to the new Surface Pro/Pen supporting this new capability, Wacom Pens that feature tilt will also “just work”! Tilt allows Windows Ink to model natural pencil sketching that response to the tilt of the pen. This support is now built into the pencil brush on the Ink Toolbar. In the above diagrams, we demonstrate how the pencil brush can be used to shade lines (on the left) and to draw arcs of varying thickness depending on the degree of tilt (on the right).

As mentioned above, tilt integration happens automatically if you use the Ink Toolbar. However, if you are not using the Windows Ink platform to render ink, and want to build your own brush that responds to tilt, you still can! There are two properties, TiltX and TiltY (respective angle of tilt against each axis of the screen plane) which are included with pointer input messages. You can access the tilt values from the PointerPointProperties included with Pointer input events, or the POINTER_PEN_INFO struct from WM_POINTER input.

These improvements automatically show up on any application that uses the Windows Ink controls, and you can be confident that we’ll continue to evolve and improve them in each release of Windows.

What’s new with Surface Dial and RadialController?

The Surface Dial introduces a new input paradigm to computing. It was designed alongside the Windows Ink experience, allowing it to truly shine when used together with a Pen. We’ve seen many experiences built to harness the new capabilities the Surface Dial brings, and are also seeing new hardware emerging, and adopting the RadialController standard. In response to your feedback, we’ve added more capabilities to the RadialController experience in the Creators Update.

First off, are some new button events for RadialControllers. These new events, Pressed and Released, combined with existing events for rotation and screen contact, will allow you to track complex interactions such as press-and-rotate or press-and-move. The example below illustrates a simple way to capture a press-and-rotate action.


_radialController.ButtonPressed += OnButtonPressed;
_radialController.ButtonReleased += OnButtonReleased;

private void OnRotationChanged(RadialController sender,
                               RadialControllerRotationChangedEventArgs args)
{
    if (args.IsButtonPressed)
    {
        /* When button is pressed, you can do modal interactions, fine-grained changes */
    }
    else
    {
        /* Otherwise, do the normal rotation behavior */
    }
}
private void SendHaptics(SimpleHapticsController hapticController)
{
    var feedbacks = hapticController.SupportedFeedback;
    foreach (SimpleHapticsControllerFeedback feedback in feedbacks)
    {
        if (feedback.Waveform ==
                    KnownSimpleHapticsControllerWaveforms.Click)
        {
            hapticController.SendHapticFeedback(feedback);
            return;
        }
    }
}

You also now have access to the Haptics engine in the Surface Dial hardware. Using SimpleHapticsController—a new object that uses the HID Simple Haptics specification—you have the power to directly send feedback to the user. You can use this to customize the feel of your menu, adding a new dimension to the experience. This object is available in the arguments of all radial controller input events.

In cases where you may want to suppress the radial menu to prevent it from blocking UI, we now have new properties ActiveControllerWhenMenuIsSuppressed and IsMenuSuppressed to let you configure when the menu is available or suppressed. When a menu is suppressed, it will not appear on press-and-hold interactions for the foreground app. Your app can listen to a new event during menu suppression to give the user an indication the menu is blocked, or build an alternate experience. Here is a code sample for this functionality:


RadialControllerConfiguration config = RadialControllerConfiguration.GetForCurrentView();
config.ActiveControllerWhenMenuIsSuppressed = myController;
config.IsMenuSuppressed = true;
  
myController.ButtonHolding += MyController_ButtonHolding;

User input running on a UI thread can sometimes lead to performance bottlenecks. With the Creator’s Update, radial controller interactions can now be handled on an off-UI thread using RadialControllerIndependentInputSource. Below is an example on how to get additional performance using this method.


RadialController controller;
Windows.UI.Input.Core.RadialControllerIndependentInputSource independentInput;
CoreApplicationView view;
            
view = CoreApplication.GetCurrentView();

var workItemHandler = new WorkItemHandler((IAsyncAction) =>
{
    independentInput = Windows.UI.Input.Core.RadialControllerIndependentInputSource.CreateForView(view);

    controller = independentInput.Controller;

    controller.RotationResolutionInDegrees = 5;

    controller.RotationChanged += Controller_RotationChanged;
    controller.ScreenContactStarted += Controller_ScreenContactStarted;
    controller.ScreenContactContinued += Controller_ScreenContactContinued;
    controller.ScreenContactEnded += Controller_ScreenContactEnded;
    controller.ControlLost += Controller_ControlLost;
    controller.ButtonClicked += Controller_ButtonClicked;
    controller.ButtonPressed += Controller_ButtonPressed;
    controller.ButtonReleased += Controller_ButtonReleased;
    controller.ButtonHolding += Controller_ButtonHolding;
    controller.ControlAcquired += Controller_ControlAcquired;

    // Begin processing input messages as they're delivered.      
    independentInput.Dispatcher.ProcessEvents(CoreProcessEventsOption.ProcessUntilQuit);
});
action = ThreadPool.RunAsync(workItemHandler, WorkItemPriority.High, WorkItemOptions.TimeSliced);

In addition to all the API additions above, you can now customize and easily add new menu items on the Radial Menu. Under “Wheel Settings” in the settings app, you can add application specific menu items that trigger keyboard combinations. Imagine customizing the controller to send your favorite shortcuts in Visual Studio, Photoshop or even when browsing the web!

The Surface Dial continues to excite users and developers alike, with these new enhancements, both developers and users have more control and flexibility in their experience. We invite you to join the numerous applications that have already delivered a great Surface Dial experience, like CorelDRAW, Autodesk’s SketchBook, Silicon Bender’s Sketchable and Algoriddim’s djay Pro. We can’t wait to see what you can do with this unique new form of input on Windows.

Join us in making Windows Ink better!

With Windows Ink and the Surface Dial additions in the Creators Update, we believe we’re just scratching the surface of what Windows Ink can do in people’s lives. Our commitment is to invest in areas that can help you innovate and remove all the barriers to our users using, loving and needing Windows Ink. This involves a spectrum of efforts, from the hardware we build by ourselves and with our partners, to the next SDK additions we make to power you app. As we continue this journey, we invite you to lend us your voice, your ideas and your feedback. Help us help you make the next great application and help us help you change the world. Tweet your ideas using #WindowsInk, email us at WindowsInk@Microsoft.com or tweet us at @WindowsInk. We would love to hear from all of you.

Thank you!

ICYMI – Your weekly TL;DR

Spring is here and so is another weekend! Check out what you might have missed this week before diving into weekend projects.

Microsoft Store Services SDK launches support for interstitial banner ads

This week we announced the launch of interstitial banner ads support in the Microsoft Store Services SDK! Read more to learn how to add them to your apps.

Simplify payments in UWP Apps with the Payment Request API from Microsoft

The Windows 10 team wants to help you take advantage of new simplified payment options for Windows 10 UWP apps. Click through right here to find out how you can add these options into your apps.

New MapControl features in Windows 10 Creators Update

We have updated the Maps platform for the Windows 10 Creators Update to give our maps a cleaner, more beautiful and realistic look so that it’s consistent between web and UWP apps.

Download Visual Studio to get started.

The Windows team would love to hear your feedback. Please keep the feedback coming using our Windows Developer UserVoice site. If you have a direct bug, please use the Windows Feedback tool built directly into Windows 10.

Desktop Bridge: Creators Update

Overview

The Creators Update is packed with new features for app developers and end users that are designed to delight developers who can offer their end users a rich experience.

User transition & data migration

Users are encouraged to download the store version of their desktop apps. If the user already has the previous desktop version of the app, the transition experience is as seamless as possible.

As a result, the Creators Update will provides a smooth transition along with some best practices for app developers to follow.

For a detailed blog post on this topic with code samples, please see this blog post.

User transition: taskbar pins & start tiles

Many users typically pin their favorite or most used apps to the taskbar pin or the start menu so they can access apps quickly.

With the Creators Update, app developers can re-route the taskbar pins and the start tile shortcuts to point to the store version of the desktop app.

User transition: file type associations & protocol handlers

The user may choose their favorite app to be the default app for a given file type or protocol. With the Creators Update, the developers can also transition the user’s choice to use the store version of the same app.

Data migration

As best practice, it is recommended that developers attempt to migrate previous user data from the desktop app upon first launch of the store version of the same app.

Users will love that they can pick up where they left off.

User transition: uninstall previous desktop app

As best practice, developers should offer uninstallation of the previous desktop app upon first launch of the store version of the app. This helps in avoiding user confusion and potential user data corruption.

Keep in mind the user can refuse the uninstallation of the previous desktop app, so the previous and store version of the app may end up running side-by-side. It is up to the app developer to decide whether or not to block the launch of the store version of the app until the previous desktop app is uninstalled.

Windows Explorer delight: previews, thumbnails, detailed properties and grouping by kind

Another focus of the Creators Update release was on user satisfaction.

Store versions of desktop apps can now take advantage of Windows Explorer perks to lead users to more frequently engage with their apps.

Preview handler

Preview handlers are triggered when an item is selected to show a lightweight, rich, read-only preview of the file’s contents in the view’s reading pane. You can do this without launching the file’s associated application.

XML sample:


<Extensions>
   <Extension Category="windows.fileTypeAssociation">
      <FileTypeAssociation Name="Foo">
         <SupportedFileTypes>
            <FileType>.bar</FileType>
         </SupportedFileTypes>
         <DesktopPreviewHandler CLSID="20000000-0000-0000-0000-000000000001" />
      </FileTypeAssociation>
   </Extension>
</Extensions>

Thumbnail handler

In Windows Explorer, thumbnails can provide a read-only view inside the file when the viewing is set to medium icons or higher.

XML sample:


<Extensions>
   <Extension Category="windows.fileTypeAssociation">
      <FileTypeAssociation Name="Foo">
         <SupportedFileTypes>
            <FileType>.bar</FileType>
         </SupportedFileTypes>
         <ThumbnailHandler CLSID="20000000-0000-0000-0000-000000000001" Cutoff="20x20" Treatment="Video Sprockets" />
      </FileTypeAssociation>
   </Extension>
</Extensions>

Property handler

Microsoft Windows Search uses property handlers to extract the values of properties from items, and uses the property system schema to determine how a specific property should be indexed. In addition, property handlers can be used to provide more details about a certain file in the properties dialog or in the details pane.

XML sample:


<Extensions>
   <Extension Category="windows.fileTypeAssociation">
      <FileTypeAssociation Name="Foo">
         <SupportedFileTypes>
            <FileType>.bar</FileType>
         </SupportedFileTypes>
         <DesktopPropertyHandler CLSID="20000000-0000-0000-0000-000000000001" />
      </FileTypeAssociation>
   </Extension>
</Extensions>

Grouping files by their ‘kind’

Developers can specify what ‘kind’ their file types are, which will enable the end users to group their files by their ‘kind’ in Windows Explorer.

XML sample:


<uap:Extension Category="windows.fileTypeAssociation">
   <uap:FileTypeAssociation Name="scoobydoo">
      <uap:SupportedFileTypes>
         <uap:FileType>.m4a</uap:FileType>
         <uap:FileType>.mta</uap:FileType>
      </uap:SupportedFileTypes>
      <rescap:KindMap>
         <rescap:Kind value="Item" />
         <rescap:Kind value="Communications" />
         <rescap:Kind value="Task" />
      </rescap:KindMap>
   </uap:FileTypeAssociation>
</uap:Extension>

Support for custom fonts installation by apps

Microsoft Windows Store apps can share their custom fonts with other Windows applications. This is done by making a few simple edits to the application manifest.

XML sample:


<Extensions>
   <uap4:Extension Category="windows.sharedFonts">
      <uap4:SharedFonts>
         <uap4:Font File="FontsJustRealize.ttf" />
         <uap4:Font File="FontsJustRealizeBold.ttf" />
      </uap4:SharedFonts>
   </uap4:Extension>
</Extensions>

Public out-of-process COM server support, aka Packaged COM

Developers can now add support for out-of-process COM and OLE extensions support for store version of desktop apps. This technology is referred to as Packaged COM. Historically, desktop apps created COM extensions that other applications could use. However, in the Windows 10 Anniversary Update release of Desktop Bridge, an application cannot expose its COM extension points as all registry entries are in its private hive and not exposed publicly to the system. Packaged COM provides a mechanism for COM and OLE entries to be declared in the manifest while the underlying subsystem handles the activation of the objects while still providing no-impact install behavior.

Firewall Rules

When apps need the users to add the app as an exception to the firewall, it translates into user confusion, additional user clicks and/or denial of exception by mistake.

The Windows Security Alert tries to educate the user, but it’s still one more decision the user must make before they can use their app.

In addition, subsequent app updates will result in the same dialog pop up and take the user through the same flow.

With the Creators Update, the developers can register for firewall rules ahead of time, so the users don’t have to make a firewall choice prior to launching their apps (including after subsequent app updates).

XML sample:


<Extensions>
   <desktop2:Extension Category="windows.firewallRules">
      <desktop2:FirewallRules Executable="foo.exe">
         <desktop2:Rule Direction="in" IPProtocol="TCP" Profile="all"/>
         <desktop2:Rule Direction="in" IPProtocol="UDP" LocalPortMin="1337" LocalPortMax="1338" Profile="domain"/>
         <desktop2:Rule Direction="in" IPProtocol="UDP" LocalPortMin="1337" LocalPortMax="1338" Profile="public"/>
         <desktop2:Rule Direction="out" IPProtocol="UDP" LocalPortMin="1339" LocalPortMax="1340" RemotePortMin="15" RemotePortMax="19" Profile="domainAndPrivate"/>
         <desktop2:Rule Direction="out" IPProtocol="GRE" Profile="private"/>
      </desktop2:FirewallRules>
   </desktop2:Extension>
</Extensions>

Other noteworthy features

  • Apps can be pre-installed
  • Messaging Application Programming Interface (MAPI) support
  • Windows App Certification Kit now includes test cases for Desktop Bridge apps
  • Use URL flag enables apps to directly open files from a URL instead of downloading a local cached version of the file

Strengthening the Microsoft Edge Sandbox

In a recent post, we outlined the layered strategy that the Microsoft Edge security team employs to protect you from vulnerabilities that could be used to compromise your device or personal data. In particular, we showed how Microsoft Edge is leveraging technologies like Code Integrity Guard (CIG) and Arbitrary Code Guard (ACG) to break some of the techniques that hackers rely on when exploiting vulnerabilities to obtain Remote Code Execution (RCE). This is where the attacker seeks to escape from web code (JS and HTML) in the browser to run native CPU code of the attacker’s choosing. This lets the attacker violate all of the browser’s rules for the web, such as same-origin policy, and so it is important to web users that we try as hard as possible to block RCE attacks.

However, despite our best efforts, sometimes attackers get RCE anyway. In this post, we’ll explore some of the significant improvements we’ve made in the Windows 10 Creators Update to strengthen our next line of defense: the Microsoft Edge sandbox.

The history of the Microsoft Edge sandbox

Because RCE can happen, and in the past, often did, browser vendors have sought to sandbox the browser to defend the rest of the user’s PC from attack. In 2007, IE7 introduced Protected Mode, the first web browser sandbox. Windows 8 added app container to the OS, primarily to support the new Windows Store app model and support the confidence promise that apps are safe to use. IE10 and IE11 leveraged app container to create EPM (Enhanced Protected Mode) a stronger browser sandbox. However, the EPM sandbox restrictions were incompatible with older ActiveX controls that predated app container, and so EPM was provided as a security enhancing option.

Microsoft Edge does not support ActiveX, so it is able to run entirely inside app container sandboxes at all times. Since the beginning, Microsoft Edge has used several app containers. At first there was a parent app container for the Manager, which created a small number of additional app containers to host content from the internet separate from intranet content. The Windows 10 Anniversary Update moved Flash into its own, separate AC. Today the Microsoft Edge app container model looks like this:

Diagram of the Microsoft app container structure. A manager app container governs individual app containers for the Internet, Intranet, Extensions, Service UI, and Adobe Flash.

The Manager provides browser features such as the URL bar, the back button, tabs, and your favorites list. The other app containers are:

  • Internet AC: hosts content from Internet sites.
  • Intranet AC: hosts content from Intranet sites. For enterprise users, that is enterprise web sites. For consumers, that is “web sites” that are control interfaces for devices on your home network, such as your Wi-Fi router, or IoT devices. Separating this RAC from the Internet e.g. protects your home Wi-Fi router from Internet attackers.
  • Extensions AC: hosts the new extensions for Microsoft Edge.
  • Flash AC: hosts the Adobe Flash player, to isolate it from the main content processes.
  • Service UI AC: hosts special web pages, such as about:flags, and the default home page.

The Internet AC is where the action is. Its job is to host web pages from anywhere, including the JS code provided by that web page, images, and multimedia. Hosting web pages is extremely complex, due to the richness of the modern web; this is the platform of the Internet, and developers need to be able to create any application and run it in this environment. Because it is complex, and hosts web pages from anywhere, this is where web security attacks begin. A malicious web site presents content intended to exploit bugs in the content hosting system, to take over control of the content process.

If an attacker gains control of an Internet AC process, they need to find some way to achieve their goals. If their goals involve compromising the user’s device or personal data stored on the device, then they’ll need to contend with escaping from the sandbox first.

Reducing the attack surface of the Microsoft Edge sandbox

One of the most effective ways to eliminate vulnerabilities in complex applications is to minimize the amount of code that an attacker can try to find vulnerabilities in. This is often referred to as attack surface reduction and it is a key tactic in our overall strategy security. To this end, Microsoft Edge in the Creators Update of Windows 10 has significantly reduced the attack surface of the sandbox by configuring the app container to further reduce its privilege.

To understand how we tuned the Microsoft Edge AC, it is first necessary to know how app container itself works. An app container process is deny-by-default for any secured object unless the object security descriptor has an allow access control entry (ACE) that would permit the app container to have access. There are three kinds of SIDs that can be used in such ACEs:

  • Capabilities: if a Capability SID based allow ACE is in the security descriptor, and the requesting AC has that Capability SID in its token, then it gets access. E.g. Webcam-related devices are accessible to processes that have the WEBCAM Capability SID such as Microsoft Edge.
  • AppID SIDs: if a specific AppID SID based allow ACE is in the security descriptor, and the requesting AC is that specific AppID, then it gets access. The main use for this is per-app storage, which uses the AppID of that app in the security descriptor, ensuring that its storage is private to that app.
  • “ALL APPLICATION PACKAGES” (“AC” SID): this is the AC wild card, allowing all app containers to access the resource. The “AC” SID exists so that Windows can provide an app platform that is useful to developers, allowing rich UWPs in the Windows Store. For instance, most of the WinRT API surface is open to the “AC” SID, so that all UWP apps can access the WinRT APIs.

The Microsoft Edge AC has been made different in only one way: the “AC” SID is not sufficient to let a requesting process access a resource. To get access, the AC must either have a matching Capability SID, or be precisely one of the named AppIDs in the security descriptor.

Thus the AC loses access to the entire WinRT API set, and every other resource that app containers normally can access. That’s great for security, with the slight problem that it would break everything about the browser, turning it into a highly secure brick.

Diagram showing the AC access scope. The Edge content process only has access to a specific subset of the AC access scope, based on capabilities.

Microsoft Edge “Tuned” AC

Diagram of a normal app container, consisting of a full AC access scope authorized by the AC SID.

“Normal” app container

So to make Microsoft Edge work again, we used specific Capabilities to light up specific areas of functionality. For instance, we created a Capability that enables COM to work. We similarly added other new capabilities, granting access to specific resources, until Microsoft Edge worked again. Then we added some telemetry to detect access control failures for the content process and shared it to users in the Windows Insider Program (WIP) to ensure that we had granted access to all resources needed for all scenarios.

What we just did here was to create a tuned sandbox for the Microsoft Edge content process, with a much tighter fit to the functional needs of the software than a normal app container provides. That is a lot of work, which is why this is not how the UWP platform works. But because web browsers are among some of the most threatened software there is, it is worth it in this case.

We repeated this work of hand-tuning a sandbox for the Flash AC, because it also is subject to a lot of attacks. Placing Flash in a tighter sandbox complements our work in this release to make Flash be click-to-run.

Making it more difficult to exploit sandbox escape vulnerabilities

The main threats in a sandbox are the broker interfaces that provide access to resources according to a defined policy. So why have brokers at all? Brokers exist to grant access according to policy, e.g. the File broker allows a website to say “upload a file by browsing your files” and the user gets to pick the file to be uploaded, without giving the web site access to all of the user’s files.

Brokers are a threat to sandboxes because brokers are code (and therefore can have bugs) and because the brokers run outside the sandbox. If an attacker can hack a broker and run code in the broker, then the attacker can escape, similar to an inmate mugging a prison guard and then putting on the guard’s uniform to escape.

The tightened Microsoft Edge content process sandbox cuts off access to many brokers (reduced attack surface) but leaves the needed brokers accessible (residual threat). An attacker might try to exploit bugs in the accessible brokers to elevate privileges and execute arbitrary native code in the broker, and thus escape. To mitigate this threat, we have applied many of the same exploit mitigation technologies to the brokers that provide service to Microsoft Edge.

Microsoft Edge also has some surprising brokers, because the exploit mitigation work forced several components to move out-of-process (OOP), including the Chakra JIT code generator and the Flash runtime process. These components had to move out of process because they necessarily do code generation (JIT) and that is not compatible with the ACG mitigation applied to the content process. Thus, these components function as the “JS code generator” broker and the “Run Flash bytecode” broker.

They can be viewed as brokers precisely because they have permission to do something that the content process is not permitted to do for itself. They are also a threat, because an attacker that can gain control of OOP JIT or the Flash sandbox could e.g. create executable code and use that for further exploitation, bypassing ACG. Because they are a threat, we have also have put these processes into tuned, less privileged app containers, again with a custom-crafted container profile built from capabilities.

Impact to attackers

Making the Microsoft Edge content process run in this tighter sandbox reduces the sandbox attack surface, but by how much? As it turns out, the reduction is quite significant:

  • 100% reduction access to MUTEXes: allow a process to lock up a resource, causing hangs.
  • 90% reduction in access to WinRT and DCOM APIs: this is the large win here, dramatically reducing Microsoft Edge’s attack surface against the WinRT API set.
  • 70% reduction access to events and symlinks: symlinks are especially interesting, because they are often used in creative bait & switch attacks to escape sandboxes.
  • 40% reduction in access to devices: Windows supports many device drivers, and their quality is somewhat beyond Microsoft’s control. The tuned sandbox cuts off access to any device that Microsoft Edge does not explicitly need, preventing attackers from using vulnerabilities in device drivers to escape, or from abusing the devices.

While attack surface reduction does not guarantee that an attacker cannot escape the sandbox, it does dramatically reduce the opportunities for attack, much like reducing the number of windows and doors in a fortress. In addition, the enabling of additional exploit mitigations for the brokers that Microsoft Edge is legitimately able to access also increases the difficulty of exploiting vulnerabilities that remain.

Conclusion

Security is a process, not a destination, so we will continue to invest in both RCE and sandbox mitigations for Microsoft Edge. These exploit mitigations combined with the strengthened sandboxing should make Microsoft Edge significantly more work for attackers to exploit, and thus discourage attackers from trying in the first place.

Crispin Cowan, Senior Program Manager, Microsoft Edge

ICYMI – Your weekly TL;DR

Busy coding weekend ahead? Before you go heads-down, get the latest from this week in Windows Developer below.

Getting Started with a Mixed Reality Platformer Using Microsoft HoloLens

The platform game genre has undergone many revolutions – and with mixed reality and HoloLens, we all have the opportunity to expand the platform game yet again. What will you build?

Windows 10 SDK Preview Build 15042 Released!

A new Windows 10 Creators Update SDK Preview was released this week! Read about what’s new in 15042.

Announcing the Xbox Live Creators Program

The Xbox Live Creators Program was announced at GDC on Wednesday, starting with an Insider Preview that gives any developer the opportunity to publish Xbox Live-enabled games on Windows 10 PCs along with Xbox One consoles. Get the details here.

Just Released – Windows Developer Evaluation Virtual Machines – February 2017 Build

And last but not least – the February 2017 edition of evaluation Windows developer virtual machines on Windows Dev Center was just released. The VMs come in Hyper-V, Parallels, VirtualBox and VMWare flavors. Get ‘em all!

Download Visual Studio to get started.

The Windows team would love to hear your feedback. Please keep the feedback coming using our Windows Developer UserVoice site. If you have a direct bug, please use the Windows Feedback tool built directly into Windows 10.

Windows Developer Day – Creators Update

Back in October, Satya, Terry and Panos introduced the Windows 10 Creators Update. They showed some of the new features and announced some amazing new additions to our Surface family of devices. Since then, my team has been working to complete the platform work for the Creators Update to deliver the full power of the Universal Windows Platform to our developers around the world.

Today, I had the chance to host the Windows Developer Day for the Creators Update and to share some of the developer-focused features that are coming soon. You can get started using these today by becoming a Windows Insider and downloading our latest preview SDK, which is now feature complete.

This is an exciting day for me because I love getting to spend time and share information with our developers. We are all driven by our mutual passion to build technology, whether we work on apps for enterprise, games or anything in between.

UWP Improvements

The Universal Windows Platform (UWP) enables your Windows apps to take advantage of the full power of the PC. To show an example of this power, I was joined on stage by Vincent Hardy from Adobe Systems, who showed us Adobe XD, a full-featured and sophisticated desktop application built on UWP.  The Adobe XD app combines beauty and power to deliver a great experience across multiple devices, form factors and input modalities.

We know that many developers want to take full advantage of modern Windows capabilities delivered in UWP, while still retaining their existing technology and code investments. The Desktop Bridge for Windows enables existing desktop applications to call UWP APIs and to be distributed and updated via the Windows Store. The Desktop Bridge makes it easier than ever to share code between Win32, .NET and UWP, taking advantage of the store’s ability to deliver easy and automated upgrades to help ensure that customers are always running the latest and best versions of their apps. Today, I showed Evernote, Photoscape X Pro and KODI, just three of the more than 330 Windows apps and games published to the store in the first four months of availability.

We believe we can work faster and smarter to deliver the best platform for developers if we work with the community. To do just this, I announced that we will make all Microsoft developer documents available and open for community collaboration. In addition, we will open our bug and feature backlog to the public and take bug contributions and feature requests starting today at noon.

In that spirit of openness, I was joined on stage by Todd Anglin from Progress, makers of the powerful Telerik controls. Telerik is popular with enterprise developers, enabling them to build great business applications with a rich set of easy-to-use UI controls. Todd showed us what these great controls can do, and announced that they are now Open Source and available on GitHub and also as NuGet packages. Find out more at telerik.com/uwp.

One of the great powers of UWP is its ability to run across a wide range of Windows devices and enable a wide range of experiences. Today, I showed a few examples of this ability:

  • Developers will be able to use the Windows Holographic Platform to bring amazing 3D experiences, both apps and games, not just to HoloLens, but to the wide range of mixed-reality headsets coming soon. We showed how easy it is to make a single app look fantastic across HoloLens and occluded headsets.
  • The new Cortana Skills Kit gives developers new and natural ways to connect user to their apps and services. We built a Cortana Skill on stage today and showed how easy it is to integrate bots created using the Microsoft Bot Framework and LUIS.
  • We know that your users interact with a variety of devices throughout the day, so we are building UWP capabilities (called “Project Rome”) to help your experiences flow seamlessly across devices, and guide users to your best and richest experience. Rome provides “apps for websites”, a simple tool that directs users to your apps (with the right context) rather than simply opening the browser. Rome’s cross-device services make it easy for users to continue an activity within your app as they move from one device to another, including Android devices. To make this possible, we released today the Rome SDK for Android.

Whenever I talk to developers, I hear that it is critically important for us to enable you to build apps that are visually beautiful and engaging, and that also take advantage of the latest and coolest hardware capabilities. It is also a high priority for us. Today, I showed some of the new capabilities that are coming to the visual and XAML layers of the platform that enable you to create beautiful user experiences through the use of effects, animations and transitions. With only a small amount of code, you can bring these effects to your apps with amazing results that delight users. In the Creators Update, we have worked to deliver new capabilities in pen and ink, including a more powerful ink toolbar that’s easy for any developer to use; smart ink, which enables recognition of lists, simple shapes and even math symbols; and enhanced APIs that enable you to do more than ever with the Surface Dial. Check out the video of the presentation for a full demo that shows a lot of these features coming together to make a beautiful and engaging app.

Games on UWP

I talked a lot about apps, but we’re also building UWP to be the best platform for building amazing games that run across the PC and Xbox One. With the Creators Update, UWP makes game development faster, easier and better. Specifically, I’d like to call out three key UWP improvements that will be appreciated by gamers and devs alike:

  • Game Mode: Windows 10 can optimize system resources to deliver the best PC gaming experience possible.
  • UWP on Xbox: UWP games (along with apps) will now be available on Xbox One via the Windows Store.
  • Windows Sonic: Microsoft’s next-gen spatial audio platform provides a powerful, immersive sound experience with support for Dolby Atmos. Windows Sonic will be available on Xbox One and Windows 10 PCs, also supporting both speakers and headphones.
  • Intelligent install: Lastly, the UWP packaging format makes it easier to install the bits in the right order, letting players jump into a new game (or app) before it has fully downloaded. And updates focus on downloading only the bits that have changed, so players can jump back in even faster.

For more information on these features and many more coming this year, tune into the Xbox sessions at GDC in a few weeks.

Windows is the home for all developers

I’m a developer, and my team is a team of developers. Like you, we love our tools and services and we’re very demanding of them. We want Windows to be the very best place to develop experiences, whether for Windows, the web or cross-platform. Today, we showed some of the performance, stability and debugging improvements in Visual Studio 2017. We demonstrated improvements to the Console and Bash, as well as to the Windows Subsystem for Linux. We showed remote access to device information via the Windows Device Portal, and even pixel-level GPU debugging with the new PIX tool for Windows.

The Windows Store team walked through several new and improved services coming in the next few months. Store Services APIs provide services to help you understand how your app is performing in the wild, including analytics and telemetry, flighting, automated submissions and updates, and ad campaigns.  The team also highlighted updated services to enable better user engagement using A/B testing, targeted push notifications and more.

And, of course, I recognize that you use lots of services from providers other than Microsoft, so today it gave me great pleasure to show powerful additions to the collection of middleware that is available to Windows Developers. Today, we announced the immediate availability to two important libraries:

  • The Windows SDK for Google Analytics brings one of the most popular and powerful real-time analytics packages to Windows. The core library works with UWP, Xamarin, Windows Forms and WPF, allowing for maximum code reuse.
  • Facebook App Install Ads are now available for UWP apps. Facebook App Install Ads are displayed in the Facebook newsfeed both on the web and in the Facebook app, bringing immense reach to Windows developers, as well as rich targeting capabilities (including interests, location and demographics) and their ads manager dashboard.

Where to get the bits

Here’s a quick recap of some of the announcements we made today:

  • Windows SDK Availability: Become a Windows Insider and get the newest feature-complete flight of the Windows 10 Creators Update SDK today.
  • Documentation: For the first time, all of Microsoft’s developer documentation will be in one place (http://docs.microsoft.com/) where it will be easier to find and use. We are also opening up the docs to community contribution.
  • Developer Platform Backlog: My team and I are making our backlog public, for both features and bugs. This will be online at noon today at https://developer.microsoft.com/windows/platform.
  • Telerik Controls: The Telerik UWP control library is now open source and available on GitHub and as NuGet packages. Find out more at telerik.com/uwp.
  • Cortana Skills Kit: The Cortana Skills Kit will enter open Developer Preview later this month.
  • Rome SDK for Android: This is a new SDK that brings the cross-devices services of “Project Rome” to Android devices, allowing easy and powerful integration with Windows. Find the new SDK at https://github.com/Microsoft/project-rome.
  • UWP Community Toolkit 1.3: Today, we are releasing a big update to the toolkit that includes many contributions from our developer community.
  • Windows SDK for Google Analytics: This new SDK, available now, enables Windows developers (UMP, Xamarin, WPF and WinForms) access to one of the more popular and powerful real-time analytics suites. Get this today at https://github.com/dotnet/windows-sdk-for-google-analytics.
  • Windows SDK for Facebook: With one billion users, Facebook offers one of the most powerful tools for you to acquire new users for your app or game. Check out the new SDK here: https://developers.facebook.com/docs/app-ads/windows-sdk.

It was a great day today, and there is more to come. We look forward to seeing you at Build in a few months when we will go deeper into the Creators Update capabilities for developers, and take a look at what’s coming next. In the meantime, I’d like to invite all of you to get started developing for Windows Creators Update today. Download the SDK, update your applications with new features (or build all-new apps that deliver the best new experiences), and please keep giving us feedback.

— Kevin

Kevin Gallo gives the developer perspective on today’s Windows 10 Event

Did you see the Microsoft Windows 10 Event this morning?  Satya, Terry, and Panos talked about some of the exciting new features coming in the Windows 10 Creators Update and announced some amazing new additions to our Surface family of devices. If you missed the event, be sure to check it out here.

As a developer, my first question when I see new features or new hardware is “What can I do with that?” We want to take advantage of the latest and coolest platform capabilities to make our apps more useful and engaging.

There were several announcements today that offer exciting opportunities for Windows developers.  Three of these that I want to tell you about are:

  • 3D in Windows 10 along with the first VR headsets capable of mixed reality through the Windows 10 Creators update.
  • Ability to put the people you care about most at the center of your experience—right where they belong—with Windows MyPeople
  • Surface Dial, a new input peripheral designed for the creative process that integrates with Windows and is complimentary to other input devices like pen. It gives developers the ability to create unique multi-modal experiences that can be customized based on context. The APIs work in both Universal Windows Platform (UWP) and Win32 apps.

Rather that write a long blog post, I decided to go down to our Channel 9 studios and record a video that gives my thoughts and provides what I hope will be a useful developer perspective on today’s announcements.  Here’s my conversation with Seth Juarez from Channel 9:

My team and I are working hard to finish the platform work that will fully support the Windows 10 Creators Update, but you can start experimenting with many of the things we talked today. Windows Insiders can download the latest flight of the SDK and get started right away.

If you want to dig deeper on the Surface Dial, check out the following links:

Stay tuned to this space for more information in the coming weeks as we get closer to the release of the Windows 10 Creator’s update.  In the meantime, we always love to hear from you and welcome your feedback at the Windows Developer Feedback site.