All posts by Lainie Huston

Meet Ear Hockey: our newest Garage project infuses inclusive design into the arcade classic – Microsoft Garage

Three years ago, a team of Microsoft employees joined together on a quest to make gaming more fun and accessible to people who are blind or have low vision. After years of hacking on machine learning algorithms in their spare time, talking with customers in the blind and low vision community, and partnering with gaming experts, the team releases their first title: Ear Hockey, a Microsoft Garage project. With in-game audio cues that enable gamers to play with or without for visuals, Ear Hockey was crafted using inclusive design to promote accessibility in game development.

When spatial audio meets accessible game development

Ear Hockey is a Pong-style, first-person paddle game, where players use their keyboard or controller to hit the ball back and forth with a partner or computer player—but with a twist on the arcade classic air hockey: it uses unique spatial audio cues, powered by Microsoft’s HRTF spatial audio engine for Unity, to signal to the player where they are in the play-space and where the ball is traveling. The team was inspired by  and worked with Daniel Kish, an expert in echolocation and President of World Access for the Blind, to craft special sounds for ball movement and contact with the paddle or other play space surfaces to help orient players. For example, a fine-tuned combination of low-frequency humming and high-frequency ticking, along with other contextual sound effects, makes the ball easier to locate with audio alone.
As they developed Ear Hockey, the team also consulted with Senior Program Manager Brannon Zahad, who has over 16 years of experience in the gaming space and works on Accessibility R&D. “One of the biggest challenges we have in the industry right now is: when a developer wants to make a game, it’s really overwhelming to think about all the different things they need to do to make their game accessible,” describes Brannon. “For the longest time, there’s been a perception that it would be impossible to build a first-person perspective game accessible to people who are blind, for example. The work this team is doing to evolve those perceptions and build technology that empowers game developers is, frankly, incredible.”
The grassroots team, which goes by Audio Augmented Reality Gaming, or AARG, originally set out to tackle making games more accessible for people who are blind or have low vision. After winning the Ability Hack category at Microsoft’s annual Hackathon in 2017, they met with several people working towards similar goals across Microsoft, including the Microsoft Researchers who developed Project Triton. The spatial audio solution creates realistic reverb effects based on objects in a video game’s map using sophisticated machine learning algorithms. The AARG team recognized the potential for this kind of audio capability for accessible game development, and has since been partnering closely to explore how this could improve not only the gamer’s experience, but also the developer’s experience in building a more inclusive game. The game studio who developed Gears of War 4, The Coalition, used Project Triton to bring next-level sound to life, featuring the possibilities of advanced reverb effects; the AARG team quickly realized they could help inspire even more innovation in this area by building their own title.

Inclusive gaming

The Ear Hockey attends the CSUN Assistive Technlogy ConferenceAARG is comprised of a passionate set of team members from every corner of the company, many of whom have never formally worked on game development. To ensure Ear Hockey was a fun experience for both people who are blind or low vision, as well as people who are sighted, the team attended one of the world’s largest accessibility summits, the CSUN Assistive Technology Conference, where they showcased a few experimental games to members of the blind and low vision community. Players of all experience types provided feedback that helped shape the game. “For me, I realized the importance of orienting the player. We’d taken for granted how critical it is to provide that information at the beginning of the experience,” shared Patrick Farley, a Content Developer in the Cloud and AI group and one of the core architects of Ear Hockey. “Since then, we’ve enhanced our tutorial with a lot more detail.” For Boris Baracaldo, a Software Engineer and the other primary game developer, he noticed there was room for more customization with respect to ambient noise. Ear Hockey gamers can now adjust the volume of background music because of the invaluable customer feedback the team received.
In addition to soliciting feedback from customers and employees from the blind and low vision community, the team also designed Ear Hockey with a visual experience, subtitles, and support for the Xbox Adaptive Controller, another Hackathon project, in an effort to make the game more inclusive for a wide range of players. This has influenced how the team approaches developing for accessibility in their day-to-day activities. “I’m a Software Developer in Dynamics and this has completely shifted my approach to making products accessible,” continued Boris, who has found new ways to apply inclusive design to the products he works on in his full-time role.

Try it Out

With Ear Hockey now available worldwide, the team is eager to get feedback from players and drive a larger conversation around accessible gaming and game development. When asked about the team’s journey over the past 3 years, original organizer and Senior Mechanical Engineer Brandon Arteaga spoke of their excitement to offer this fun experience and help promote awareness about the power of echolocation. “We support Daniel Kish’s mission to teach echolocation as a skillset which can empower members of the blind and low-vision community to explore the world more freely. Anything we can do to shine a light on his work would be a dream come true.” Wilson Dreewes, a Senior Software Developer in AARG added, “Our goal is to showcase feature sets and what can be done in this space. We hope that the games we’re releasing now would be fun in and of themselves, but also that they inspire more innovation in game development.” So, have fun playing! Please try out Ear Hockey and share any feedback to the team on the game or accessible game development via UserVoice. You can also read about the full game features on the Garage Workbench.

Go to Original Article
Author: Steve Clarke

Meet Ear Hockey: our newest Garage project infuses inclusive design into the arcade classic – Microsoft Garage

Three years ago, a team of Microsoft employees joined together on a quest to make gaming more fun and accessible to people who are blind or have low vision. After years of hacking on machine learning algorithms in their spare time, talking with customers in the blind and low vision community, and partnering with gaming experts, the team releases their first title: Ear Hockey, a Microsoft Garage project. With in-game audio cues that enable gamers to play with or without for visuals, Ear Hockey was crafted using inclusive design to promote accessibility in game development.

When spatial audio meets accessible game development

Ear Hockey is a Pong-style, first-person paddle game, where players use their keyboard or controller to hit the ball back and forth with a partner or computer player—but with a twist on the arcade classic air hockey: it uses unique spatial audio cues, powered by Microsoft’s HRTF spatial audio engine for Unity, to signal to the player where they are in the play-space and where the ball is traveling. The team was inspired by  and worked with Daniel Kish, an expert in echolocation and President of World Access for the Blind, to craft special sounds for ball movement and contact with the paddle or other play space surfaces to help orient players. For example, a fine-tuned combination of low-frequency humming and high-frequency ticking, along with other contextual sound effects, makes the ball easier to locate with audio alone.
As they developed Ear Hockey, the team also consulted with Senior Program Manager Brannon Zahad, who has over 16 years of experience in the gaming space and works on Accessibility R&D. “One of the biggest challenges we have in the industry right now is: when a developer wants to make a game, it’s really overwhelming to think about all the different things they need to do to make their game accessible,” describes Brannon. “For the longest time, there’s been a perception that it would be impossible to build a first-person perspective game accessible to people who are blind, for example. The work this team is doing to evolve those perceptions and build technology that empowers game developers is, frankly, incredible.”
The grassroots team, which goes by Audio Augmented Reality Gaming, or AARG, originally set out to tackle making games more accessible for people who are blind or have low vision. After winning the Ability Hack category at Microsoft’s annual Hackathon in 2017, they met with several people working towards similar goals across Microsoft, including the Microsoft Researchers who developed Project Triton. The spatial audio solution creates realistic reverb effects based on objects in a video game’s map using sophisticated machine learning algorithms. The AARG team recognized the potential for this kind of audio capability for accessible game development, and has since been partnering closely to explore how this could improve not only the gamer’s experience, but also the developer’s experience in building a more inclusive game. The game studio who developed Gears of War 4, The Coalition, used Project Triton to bring next-level sound to life, featuring the possibilities of advanced reverb effects; the AARG team quickly realized they could help inspire even more innovation in this area by building their own title.

Inclusive gaming

The Ear Hockey attends the CSUN Assistive Technlogy ConferenceAARG is comprised of a passionate set of team members from every corner of the company, many of whom have never formally worked on game development. To ensure Ear Hockey was a fun experience for both people who are blind or low vision, as well as people who are sighted, the team attended one of the world’s largest accessibility summits, the CSUN Assistive Technology Conference, where they showcased a few experimental games to members of the blind and low vision community. Players of all experience types provided feedback that helped shape the game. “For me, I realized the importance of orienting the player. We’d taken for granted how critical it is to provide that information at the beginning of the experience,” shared Patrick Farley, a Content Developer in the Cloud and AI group and one of the core architects of Ear Hockey. “Since then, we’ve enhanced our tutorial with a lot more detail.” For Boris Baracaldo, a Software Engineer and the other primary game developer, he noticed there was room for more customization with respect to ambient noise. Ear Hockey gamers can now adjust the volume of background music because of the invaluable customer feedback the team received.
In addition to soliciting feedback from customers and employees from the blind and low vision community, the team also designed Ear Hockey with a visual experience, subtitles, and support for the Xbox Adaptive Controller, another Hackathon project, in an effort to make the game more inclusive for a wide range of players. This has influenced how the team approaches developing for accessibility in their day-to-day activities. “I’m a Software Developer in Dynamics and this has completely shifted my approach to making products accessible,” continued Boris, who has found new ways to apply inclusive design to the products he works on in his full-time role.

Try it Out

With Ear Hockey now available worldwide, the team is eager to get feedback from players and drive a larger conversation around accessible gaming and game development. When asked about the team’s journey over the past 3 years, original organizer and Senior Mechanical Engineer Brandon Arteaga spoke of their excitement to offer this fun experience and help promote awareness about the power of echolocation. “We support Daniel Kish’s mission to teach echolocation as a skillset which can empower members of the blind and low-vision community to explore the world more freely. Anything we can do to shine a light on his work would be a dream come true.” Wilson Dreewes, a Senior Software Developer in AARG added, “Our goal is to showcase feature sets and what can be done in this space. We hope that the games we’re releasing now would be fun in and of themselves, but also that they inspire more innovation in game development.” So, have fun playing! Please try out Ear Hockey and share any feedback to the team on the game or accessible game development via UserVoice. You can also read about the full game features on the Garage Workbench.

Go to Original Article
Author: Steve Clarke

MileIQ Team Launches New Microsoft Garage Project to Simplify Expense Management – Microsoft Garage

In the fall of 2015, a startup team committed to making mileage logging easier joined Microsoft to expand their ability to empower the self-directed worker. Today, the team that created MileIQ now announces an iOS app to simplify expense management. Spend, a Microsoft Garage project, is now available for download in the US App Store.

The Microsoft Garage has released dozens of projects in the four years following the October 2014 launch of its Garage project program. Several projects have gone on to become features of flagship Microsoft products or new, branded products in their own right. Both self-organized, grassroots teams and arms of Microsoft product groups have leveraged the program to collect user feedback, add new features, and refine their approach.

Spend joins a host of Garage projects that have narrowed in on a specific target customer to understand their needs and build scenarios that tackle acute problems. Sports Performance Platform worked with the Seattle Seahawks, Benfica, Cricket Australia, and Real Madrid among other major sports teams and organizations. Video Indexer—the once-titled Video Breakdown and Garage project alum—synthesized multiple Microsoft Cognitive Services into an improved experience and is now offered as a standalone Azure Media Service. Microsoft Kaizala began its journey focusing on mobile collaboration scenarios, getting its big break working with the Indian state government to organize a massive holiday event attended by over 20 million people.

Hassle-Free, On-the-go Expense Management

Spend is mobile-first and built with the user in mind, making it simple to track expenses for reimbursements or taxes

• Quickly manage all your purchases for your expense reports with automatically tracked expenses from a connected credit card, debit card or bank account
• See purchases in a feed and easily classify expenses as business or personal with a single swipe
• Edit purchases or bulk classify expenses through the web dashboard
• Create accurate reports with only a few clicks for the week, month, or another customer period
• Fully customize reports which are available in either spreadsheet or PDF, commonly-used formats compatible with leading accounting and expense management software
• Snap pictures of receipts and attached to purchase with additional features for easy cash purchase management
• Track confidently: Spend uses 256-bit encryption, bank-level security, with Microsoft certifications

Spend, a Microsoft Garage project

The team has long focused on creating solutions that simplify work and empower self-directed workers, and built its first product MileIQ to enhance the mileage logging experience. When you’re on the road a lot, it can be time-consuming and chaotic to organize mileage and gas information for reimbursements or tax deductions. Now, the team is also turning its attention to another pain point for this audience: expense management.

“Keeping, sorting, and tracking paper receipts is annoying and inefficient. Spend uses intelligent features to bring receipt and expense tracking to the modern era,” says Heman Chawda, Product Manager for Spend. Built for mobile and designed around those always on the go, Spend takes a fresh approach to expense management. “It’s a great opportunity for us to be able to explore this space,” shared Nat Robinson, General Manager of MileIQ. “We are a growing team at Microsoft working on a number of experiences that empower small and medium businesses. The Microsoft Garage gives us the chance to offer a simplified expense management experience and really hone the value we provide. This is an important area of investment for us—we’re really excited to take this first step.”

MileIQ Team Launches New Microsoft Garage Project to Simplify Expense Management – Microsoft Garage

In the fall of 2015, a startup team committed to making mileage logging easier joined Microsoft to expand their ability to empower the self-directed worker. Today, the team that created MileIQ now announces an iOS app to simplify expense management. Spend, a Microsoft Garage project, is now available for download in the US App Store.

The Microsoft Garage has released dozens of projects in the four years following the October 2014 launch of its Garage project program. Several projects have gone on to become features of flagship Microsoft products or new, branded products in their own right. Both self-organized, grassroots teams and arms of Microsoft product groups have leveraged the program to collect user feedback, add new features, and refine their approach.

Spend joins a host of Garage projects that have narrowed in on a specific target customer to understand their needs and build scenarios that tackle acute problems. Sports Performance Platform worked with the Seattle Seahawks, Benfica, Cricket Australia, and Real Madrid among other major sports teams and organizations. Video Indexer—the once-titled Video Breakdown and Garage project alum—synthesized multiple Microsoft Cognitive Services into an improved experience and is now offered as a standalone Azure Media Service. Microsoft Kaizala began its journey focusing on mobile collaboration scenarios, getting its big break working with the Indian state government to organize a massive holiday event attended by over 20 million people.

Hassle-Free, On-the-go Expense Management

Spend is mobile-first and built with the user in mind, making it simple to track expenses for reimbursements or taxes

• Quickly manage all your purchases for your expense reports with automatically tracked expenses from a connected credit card, debit card or bank account
• See purchases in a feed and easily classify expenses as business or personal with a single swipe
• Edit purchases or bulk classify expenses through the web dashboard
• Create accurate reports with only a few clicks for the week, month, or another customer period
• Fully customize reports which are available in either spreadsheet or PDF, commonly-used formats compatible with leading accounting and expense management software
• Snap pictures of receipts and attached to purchase with additional features for easy cash purchase management
• Track confidently: Spend uses 256-bit encryption, bank-level security, with Microsoft certifications

Spend, a Microsoft Garage project

The team has long focused on creating solutions that simplify work and empower self-directed workers, and built its first product MileIQ to enhance the mileage logging experience. When you’re on the road a lot, it can be time-consuming and chaotic to organize mileage and gas information for reimbursements or tax deductions. Now, the team is also turning its attention to another pain point for this audience: expense management.

“Keeping, sorting, and tracking paper receipts is annoying and inefficient. Spend uses intelligent features to bring receipt and expense tracking to the modern era,” says Heman Chawda, Product Manager for Spend. Built for mobile and designed around those always on the go, Spend takes a fresh approach to expense management. “It’s a great opportunity for us to be able to explore this space,” shared Nat Robinson, General Manager of MileIQ. “We are a growing team at Microsoft working on a number of experiences that empower small and medium businesses. The Microsoft Garage gives us the chance to offer a simplified expense management experience and really hone the value we provide. This is an important area of investment for us—we’re really excited to take this first step.”

MileIQ Team Launches New Microsoft Garage Project to Simplify Expense Management – Microsoft Garage

In the fall of 2015, a startup team committed to making mileage logging easier joined Microsoft to expand their ability to empower the self-directed worker. Today, the team that created MileIQ now announces an iOS app to simplify expense management. Spend, a Microsoft Garage project, is now available for download in the US App Store.

The Microsoft Garage has released dozens of projects in the four years following the October 2014 launch of its Garage project program. Several projects have gone on to become features of flagship Microsoft products or new, branded products in their own right. Both self-organized, grassroots teams and arms of Microsoft product groups have leveraged the program to collect user feedback, add new features, and refine their approach.

Spend joins a host of Garage projects that have narrowed in on a specific target customer to understand their needs and build scenarios that tackle acute problems. Sports Performance Platform worked with the Seattle Seahawks, Benfica, Cricket Australia, and Real Madrid among other major sports teams and organizations. Video Indexer—the once-titled Video Breakdown and Garage project alum—synthesized multiple Microsoft Cognitive Services into an improved experience and is now offered as a standalone Azure Media Service. Microsoft Kaizala began its journey focusing on mobile collaboration scenarios, getting its big break working with the Indian state government to organize a massive holiday event attended by over 20 million people.

Hassle-Free, On-the-go Expense Management

Spend is mobile-first and built with the user in mind, making it simple to track expenses for reimbursements or taxes

• Quickly manage all your purchases for your expense reports with automatically tracked expenses from a connected credit card, debit card or bank account
• See purchases in a feed and easily classify expenses as business or personal with a single swipe
• Edit purchases or bulk classify expenses through the web dashboard
• Create accurate reports with only a few clicks for the week, month, or another customer period
• Fully customize reports which are available in either spreadsheet or PDF, commonly-used formats compatible with leading accounting and expense management software
• Snap pictures of receipts and attached to purchase with additional features for easy cash purchase management
• Track confidently: Spend uses 256-bit encryption, bank-level security, with Microsoft certifications

Spend, a Microsoft Garage project

The team has long focused on creating solutions that simplify work and empower self-directed workers, and built its first product MileIQ to enhance the mileage logging experience. When you’re on the road a lot, it can be time-consuming and chaotic to organize mileage and gas information for reimbursements or tax deductions. Now, the team is also turning its attention to another pain point for this audience: expense management.

“Keeping, sorting, and tracking paper receipts is annoying and inefficient. Spend uses intelligent features to bring receipt and expense tracking to the modern era,” says Heman Chawda, Product Manager for Spend. Built for mobile and designed around those always on the go, Spend takes a fresh approach to expense management. “It’s a great opportunity for us to be able to explore this space,” shared Nat Robinson, General Manager of MileIQ. “We are a growing team at Microsoft working on a number of experiences that empower small and medium businesses. The Microsoft Garage gives us the chance to offer a simplified expense management experience and really hone the value we provide. This is an important area of investment for us—we’re really excited to take this first step.”

MileIQ Team Launches New Microsoft Garage Project to Simplify Expense Management – Microsoft Garage

In the fall of 2015, a startup team committed to making mileage logging easier joined Microsoft to expand their ability to empower the self-directed worker. Today, the team that created MileIQ now announces an iOS app to simplify expense management. Spend, a Microsoft Garage project, is now available for download in the US App Store.

The Microsoft Garage has released dozens of projects in the four years following the October 2014 launch of its Garage project program. Several projects have gone on to become features of flagship Microsoft products or new, branded products in their own right. Both self-organized, grassroots teams and arms of Microsoft product groups have leveraged the program to collect user feedback, add new features, and refine their approach.

Spend joins a host of Garage projects that have narrowed in on a specific target customer to understand their needs and build scenarios that tackle acute problems. Sports Performance Platform worked with the Seattle Seahawks, Benfica, Cricket Australia, and Real Madrid among other major sports teams and organizations. Video Indexer—the once-titled Video Breakdown and Garage project alum—synthesized multiple Microsoft Cognitive Services into an improved experience and is now offered as a standalone Azure Media Service. Microsoft Kaizala began its journey focusing on mobile collaboration scenarios, getting its big break working with the Indian state government to organize a massive holiday event attended by over 20 million people.

Hassle-Free, On-the-go Expense Management

Spend is mobile-first and built with the user in mind, making it simple to track expenses for reimbursements or taxes

• Quickly manage all your purchases for your expense reports with automatically tracked expenses from a connected credit card, debit card or bank account
• See purchases in a feed and easily classify expenses as business or personal with a single swipe
• Edit purchases or bulk classify expenses through the web dashboard
• Create accurate reports with only a few clicks for the week, month, or another customer period
• Fully customize reports which are available in either spreadsheet or PDF, commonly-used formats compatible with leading accounting and expense management software
• Snap pictures of receipts and attached to purchase with additional features for easy cash purchase management
• Track confidently: Spend uses 256-bit encryption, bank-level security, with Microsoft certifications

Spend, a Microsoft Garage project

The team has long focused on creating solutions that simplify work and empower self-directed workers, and built its first product MileIQ to enhance the mileage logging experience. When you’re on the road a lot, it can be time-consuming and chaotic to organize mileage and gas information for reimbursements or tax deductions. Now, the team is also turning its attention to another pain point for this audience: expense management.

“Keeping, sorting, and tracking paper receipts is annoying and inefficient. Spend uses intelligent features to bring receipt and expense tracking to the modern era,” says Heman Chawda, Product Manager for Spend. Built for mobile and designed around those always on the go, Spend takes a fresh approach to expense management. “It’s a great opportunity for us to be able to explore this space,” shared Nat Robinson, General Manager of MileIQ. “We are a growing team at Microsoft working on a number of experiences that empower small and medium businesses. The Microsoft Garage gives us the chance to offer a simplified expense management experience and really hone the value we provide. This is an important area of investment for us—we’re really excited to take this first step.”

MileIQ Team Launches New Microsoft Garage Project to Simplify Expense Management – Microsoft Garage

In the fall of 2015, a startup team committed to making mileage logging easier joined Microsoft to expand their ability to empower the self-directed worker. Today, the team that created MileIQ now announces an iOS app to simplify expense management. Spend, a Microsoft Garage project, is now available for download in the US App Store.

The Microsoft Garage has released dozens of projects in the four years following the October 2014 launch of its Garage project program. Several projects have gone on to become features of flagship Microsoft products or new, branded products in their own right. Both self-organized, grassroots teams and arms of Microsoft product groups have leveraged the program to collect user feedback, add new features, and refine their approach.

Spend joins a host of Garage projects that have narrowed in on a specific target customer to understand their needs and build scenarios that tackle acute problems. Sports Performance Platform worked with the Seattle Seahawks, Benfica, Cricket Australia, and Real Madrid among other major sports teams and organizations. Video Indexer—the once-titled Video Breakdown and Garage project alum—synthesized multiple Microsoft Cognitive Services into an improved experience and is now offered as a standalone Azure Media Service. Microsoft Kaizala began its journey focusing on mobile collaboration scenarios, getting its big break working with the Indian state government to organize a massive holiday event attended by over 20 million people.

Hassle-Free, On-the-go Expense Management

Spend is mobile-first and built with the user in mind, making it simple to track expenses for reimbursements or taxes

• Quickly manage all your purchases for your expense reports with automatically tracked expenses from a connected credit card, debit card or bank account
• See purchases in a feed and easily classify expenses as business or personal with a single swipe
• Edit purchases or bulk classify expenses through the web dashboard
• Create accurate reports with only a few clicks for the week, month, or another customer period
• Fully customize reports which are available in either spreadsheet or PDF, commonly-used formats compatible with leading accounting and expense management software
• Snap pictures of receipts and attached to purchase with additional features for easy cash purchase management
• Track confidently: Spend uses 256-bit encryption, bank-level security, with Microsoft certifications

Spend, a Microsoft Garage project

The team has long focused on creating solutions that simplify work and empower self-directed workers, and built its first product MileIQ to enhance the mileage logging experience. When you’re on the road a lot, it can be time-consuming and chaotic to organize mileage and gas information for reimbursements or tax deductions. Now, the team is also turning its attention to another pain point for this audience: expense management.

“Keeping, sorting, and tracking paper receipts is annoying and inefficient. Spend uses intelligent features to bring receipt and expense tracking to the modern era,” says Heman Chawda, Product Manager for Spend. Built for mobile and designed around those always on the go, Spend takes a fresh approach to expense management. “It’s a great opportunity for us to be able to explore this space,” shared Nat Robinson, General Manager of MileIQ. “We are a growing team at Microsoft working on a number of experiences that empower small and medium businesses. The Microsoft Garage gives us the chance to offer a simplified expense management experience and really hone the value we provide. This is an important area of investment for us—we’re really excited to take this first step.”

The Napkin Disrupted: Meet Ink to Code, a Microsoft Garage Project – Microsoft Garage

Urban legend has it that some of the greatest ideas in history started with a napkin. The Gettysburg Address, the poem that gave way to the U.S. National Anthem, and the premise of the Harry Potter series—each were reportedly born into the world through the medium of sketches on scrap paper—and when app creators put pen to paper for their ideas, this is often a canvas of choice. While rapid prototyping with the napkin and the whiteboard holds its charms, less charming is the prospect of translating quick sketches into working code.

Last summer, a group of Garage interns tackled this problem by creating a prototype of their own: meet Ink to Code, a Microsoft Garage project, now available in the United States and Canada. Ink to Code is a Windows app that enables developers to draw wire frame sketches and export them into Visual Studio, expediting the process of prototyping Universal Windows Platform (UWP) and Android user interfaces.

The Garage Internship takes a unique, entrepreneurial spin on the traditional big tech model; rather than embedding with a full-time organization, students work in groups of 5-6 as a distinct team, building their own, standalone project. Microsoft product groups vie for intern teams to work on proposed projects by pitching opportunities to interns at the start of the internship. This summer at the Microsoft New England Research and Development facility (fondly known as NERD) located in Cambridge, MA, 6 interns found their passion in the pitch for Ink to Code and signed up to work with the Xamarin team sponsoring the idea. 5 more interns studying at MIT joined the Garage team to continue working on the project.

Building a Better Napkin

Ink to Code Guide Feature Screenshot
Ink to Code captures sketches of basic visual elements and translates them into the beginnings of an app in Visual Studio

The sponsoring team and interns were both motivated by a desire to modernize the brainstorming and prototyping process from using napkin and white board sketches, to an experience that is more automated and cohesive with the Visual Studio suite. “We’ve all been in that situation as developers,” notes Alex Corrado, a Senior Software Engineer on the Xamarin Designer team, and one of the originators of the project. “Getting your ideas for a new app or feature onto paper is one of the fastest, most natural parts of the brainstorming. But then, you ultimately need to turn that sketch into code and sooner than you know it, 10, 20, 30 iterations of a sketch really add up.” The team turned to the Smart Ink built into UWP to preserve the natural desire to sketch, while bridging the gap between analog and digital with a companion app for Visual Studio. In the Windows 10 Fall Creators Update, Smart Ink improves ink recognition with AI. The Ink to Code team leveraged this machine learning technology to save months on development time.

Ink to Code translates common design symbols into the beginnings of an app in Visual Studio. The first version supports basic app visual elements including labels, text fields, text paragraphs, images, and buttons. While Ink to Code can’t bring a full app vision to life, it significantly cuts down on creating the basic foundation of the app with the power of automation. Perhaps even more valuable is the way it enables developers and designers to collaborate differently. Ink to Code can be used as a more productive canvas in brainstorm meetings, or even more significantly, as a tool that can bridge the gap between collaborators with different levels of design or technical knowledge.

A Prototype for Prototypes

A core part of the Garage intern experience is conducting customer development and research, and the Ink to Code team worked with internal developers and designers to get feedback on their prototype. Today, the sponsoring Xamarin team releases the app to drastically expand the pool of feedback. Alex also shares, “Our goal is to hear from a wide variety of app creators, so we know what people like most and what we should add.”

“Developers are crazy diverse, and no experience could serve them 100% on day 1, but their feedback can help us get closer, faster,” adds George Matthews a Senior Program Manager in the Garage as well as a key originator of Ink to Code. The gut reaction of any app creator is to make sure your project is polished and perfect before shipping it, especially when releasing to an audience of your developer peers. The Ink to Code team is embracing the mindset of getting feedback early, and developing with the customer and for the customers.  George continues, “The feedback from our first customers will really help us stack rank our backlog.”

To check out Ink to Code and feed into the future direction of the project, download it at the Microsoft Store and share your thoughts via in-app feedback or UserVoice. Ink to Code is best with Visual Studio 2017.