Category Archives: Screen Capture

Auto Added by WPeMatico

New Ways to do Screen Capture – Windows Developer Blog

Screen capture supports scenarios like screen recording for eLearning, screen sharing for collaboration, game streaming, remote diagnostics, and taking screen shots for visual comparison or editing. The new UWP WindowsGraphicsCapture APIs provide a modern, performant way of capturing screen contents in Win32 and UWP applications.WindowsGraphicsCapture APIs first shipped in the Windows 10 April 2018 Update (1803). These APIs were built for developers who depended on screen capture functionality for their modern applications without depending on restricted capabilities. These APIs enable capture of application windows, displays, and environments in a secure, easy to use way with the use of a system picker UI control.

When an application makes a capture request, it is presented to the user in the form of a control where the user can decide what visuals (displays or applications) they’ll allow the application to capture. Because the operating system manages this experience, UWP applications cannot spoof a request for access to windows outside of the capturing application’s process.

System Picker for Capture Selection
Once the capture is initiated, the visual being captured is enhanced with a small yellow border to remind the user of what is being shared and/or possibly recorded.

Yellow Boarder Capture Indicator
You can see an example of the WindowsGraphicsCapture APIs paired with the Windows.Media.Transcoding APIs in this simple Screen Recorder demo.

For the Windows 10 October Update (1809), we delivered a convenience feature that enables applications to capture any child visual they “own.”
CreateFromVisual allows developers to do various things with their existing content:
Save snapshots of visual trees (similar to RenderTargetBitmap)
Save a stream of frames from their visual trees (can be hooked up to WinRT encoding APIs to save video)
Apply expensive effects that are not supported by the compositor
Because the application owns its content (by virtue of having access to the visuals), no dialog or consent is required. Instead, developers can construct a GraphicsCaptureItem that represents a visual by using a static method.

A common request from Win32 developers was better interoperability between Windows Graphics Capture and HWNDs.
For the Windows 10 May 2019 Update our capture team’s engineering efforts went to support Win32 interop scenarios with two new APIs. Now Win32 applications who use screen capture features can use modern APIs that create capture items they’re familiar with.
These API extensions allow the graphics capture API to target a single window or monitor given its ntuser handles (HWND and HMONITOR). It otherwise operates identically to the WinRT-clean version of the API that receives a capture item from the Capture Picker UI. These APIs are available in the Windows 10 May 2019 Update (look in the Windows.Graphics.Capture.Interop.h header).
Samples for WPF and Win32 screen capture are available at the Windows.UI.Composition-Win32-Samples GitHub.

IsCursorEnabled
Windows insiders who have opted in to skip ahead/fast rings and are running build 18994 or greater will see a new API IsCursorEnabled which supports omitting the cursor from capture. Please see all the warnings that apply to APIs in these environments.
ExcludeFromCapture
We have added a new flag to the SetWindowDisplayAffinity function, which supports content protection by returning black in screen captures of these windows. Sometimes, applications want to simply exclude a window from capture and not return black because things like recording buttons are not always desirable in captured content. When the new flag, WDA_EXCLUDEFROMCAPTURE, is set, the window will be removed from capture entirely.

If you have feedback or want to get in touch with the capture team, you can send us email at [email protected] If you want to stay up to date on the latest features from our team, follow us on Twitter @WindowsUI.

Bringing Screen Capture to Microsoft Edge with the Media Capture API

Beginning with the EdgeHTML 17, Microsoft Edge is the first browser to support Screen Capture via the Screen Capture API. Web developers can start building on this feature today by upgrading to the Windows 10 April 2018 Update, or using one of our free virtual machines.
Screen Capture uses the new getDisplayMedia API specified by the W3C Web Real-Time Communications Working Group The feature lets web pages capture output of a user’s display device, commonly used to broadcast a desktop for plugin-free virtual meetings or presentations. Using Media Capture, Microsoft Edge can capture all Windows applications–including including Win32 and Universal Windows Platform applications (UWP apps).
In this post, we’ll walk through how Screen Capture is implemented in Microsoft Edge, and what’s on our roadmap for future releases, as well as some best practices for developers looking to get started with this API today.
Getting started with the Screen Capture API
The getDisplayMedia() method is the heart of the Screen Capture API. The getDisplayMedia() call takes MediaStreamConstraints as an optional input argument.  Once the user grants permission, the getDisplayMedia() call will return a promise with a MediaStream object representing the user-selected capture device.
The MediaStream object will only have a MediaStreamTrack for the captured video stream; there is no MediaStreamTrack corresponding to a captured audio stream. The MediaStream object can be rendered on multiple rendering targets, for example, by setting it on the srcObject attribute of MediaElement (e.g. video tags).
While the operation of the getDisplayMedia API is superficially very similar to getUserMedia, there are some important differences. To ensure users are in control of any sensitive information which may be captured, getDisplayMedia does not allow the MediaStreamConstraints argument to influence the selection of sources. This is different from getUserMedia, which enables picking a specific capture device.
Our implementation of Screen Capture currently does not support the use of MediaStreamConstraints to influence MediaStreamTrack characteristics (such as framerate or resolution). The getSettings() method can’t be used to obtain the type of display surface that was captured, although information such as the width, height, aspect ratio and framerate of the capture can be obtained. Within the W3C Web Real-Time Communications Working Group there is ongoing discussion of how MediaStreamConstraints influences properties of the captured screen device, such as resolution and framerate, but consensus has not yet been reached.
User permissions
While screen capture functionality can enable a lot of exciting user and business scenarios, removing the need for additional third-party software, plugins, or manual user steps for scenarios such as conference calls and desktop screenshots, it also introduces security and privacy concerns. Explicit, opt-in user consent is a critical part of the feature.
While the W3C specification recommends some best practices, it also leaves each browser some flexibility in implementation. To balance security and privacy concerns and user experiences, our implementation requires the following:
An HTTPS origin is required for getDisplayMedia() to be called.
The user is prompted to allow or deny permission to allow screen capture when getDisplayMedia() is called.
While the user’s chosen permissions persist, the capture picker UI will come up for each getDisplayMedia() call. Permissions can be managed via the site permissions UI in Microsoft Edge (in Settings or via the site info panel in the URL bar).
If a webpage calls getDisplayMedia() from an iframe, we will manage the screen capture device permission separately based on its own URL. This provides protection to the user in cases where the iframe is from a different domain than its parent webpage.
As noted above, we do not permit MediaStreamConstraints to influence the selection of getDisplayMedia screen capture sources.
Sample scenarios using screen capture
Screen capture is an essential step in many scenarios, including real-time audio and video communications. Below we walk through a simple scenario introducing you to how to use the Screen Capture functionality.
Capture photo from a screen capture device
Let’s assume we have a video tag on the page and it is set to autoplay.  Prior to calling navigator.getDisplayMedia, we set up constraints and create a handleSuccess function to wire the screen capture stream to the video tag as well as a handleError function to log an error to the console if one occurs.
https://gist.github.com/kypflug/5d1e15bb33beb008a91a8bc789250255
When navigator.getDisplayMedia is called, the picker UI comes up and the user can select whether to share a window or a display.
The Picker UI allows the user to select whether to share the entire display, or a particular window.
While being captured, the chosen application or display will have a yellow border draw around it which is not included in the capture frame. Application windows being captured will return black frames while minimized (though they will still be enumerated in the picker); if the window is restored, rendering will resume.
If an application window includes a privacy flag (setDisplayAffinity or isScreenCaptureEnabled) the application is not enumerated in the picker. Application windows being captured will not include overlapping content, which is an improvement on snapshotting the entire display and cropping to window location.
What’s next for Screen Capture
Currently the MediaStream produced by getDisplayMedia can be consumed by the ORTC API in Microsoft Edge.  To optimize encoding in screen capture scenarios, the  degradationPreference encoding parameter is used.  For applications where video motion is limited (e.g. a slideshow presentation), degradationPreference should be set to “maintain-resolution” for best results. To limit the maximum framerate that can be sent over the wire, the maxFramerate encoding parameter can be used.
To use the MediaStream with the WebRTC 1.0 API in Microsoft Edge, we recommend the adapter.js library, as we work towards support for getDisplayMedia along with the WebRTC 1.0 object model in a future release.
You can get started with the Screen Capture API in Microsoft Edge today on EdgeHTML 17.17134 or higher, available in the Windows 10 April 2018 Update or through the free virtual machines on the Microsoft Edge Developer Site. Try it out and let us know what you think by reaching out to @MSEdgeDev on Twitter or submitting feedback at https://issues.microsoftedge.com!
– Angelina Gambo, Senior Program Manager, Microsoft Edge
– Bernard Aboba, Principal Architect, Skype