Tag Archives: arms

Why Would Prosthetic Arms Need to See or Connect to Cloud AI?

Based on “Connected Arms”, a keynote talk at the O’Reilly AI Conference delivered by Joseph Sirosh, CTO for AI at Microsoft. Content reposted from this O’Reilly Media website.

There are over 1 million new amputees every year, i.e. one every 30 seconds – a truly shocking statistic.

The World Health Organization estimates that between 30 to 100 million people around the world are living with limb loss today. Unfortunately, only 5-15% of this population has access to prosthetic devices.

Although prostheses have been around since ancient times, their successful use has been severely limited for millennia by several factors, with cost being the major one. Although it is possible to get sophisticated bionic arms today, the cost of such devices runs into tens of thousands of dollars. These devices are just not widely available today. What’s more, having these devices interface satisfactorily with the human body has been a massive issue, partly due to the challenges of working with the human nervous system. Such devices generally need to be tailored to work with each individual’s nervous system, a process that often requires expensive surgery.

Is it possible for a new generation of human beings to finally help us break through these long-standing barriers?

Can prosthetic devices learn to adapt to us, as opposed to the other way around?

A Personalized Prosthetic Arm for $100?

In his talk, Joseph informs us about how, using the combination of:

  • Low-cost off-the-shelf electronics,
  • 3D-printing, and
  • Cloud AI, for intelligent, learned, personalized behavior,

it is now becoming possible to deliver prosthetic arms at a price point of around $100.

Joseph takes the smartARM as an example of such a breakthrough device. A prototype built by two undergraduate students from Canada who recently won the first prize in Microsoft’s Imagine Cup, the smartARM is 3D-printed, has a camera in the palm of its hand and is connected to the cloud. The magic is in the cloud, where a computer vision service recognizes the objects seen by the camera. Deep learning algorithms then generate the precise finger movements needed to grasp the object near the arm. Essentially, the cloud vision service classifies the object and generates the right grip or action, such as a pincer action to pick up a bunch of keys on a ring, or a palmar action to pick up a wineglass. The grip itself is a learned behavior which can be trained and customized.

The user of the prosthetic arm triggers the grip (or its release) by flexing any muscle of their choice on their body, for instance, their upper arm muscle. A myoelectric sensor located in a band that is strapped over that muscle detects the signal and triggers the grip or its release.

Simple, Adaptable Architecture

The architecture of this grip classification solution is shown below. The input to the raspberry pi on the smartARM comes from camera and the muscle sensor. These inputs are sent to the Azure Custom Vision Service, an API in the cloud which has been trained on grip classifications and is able to output the appropriate grip. This grip is sent back to an Arduino board in the smartARM which can then trigger the servo motors that realize that grip in the physical world, i.e. as soon as the smartARM gets the signal to do so from the muscle sensor.

This is an adaptable architecture. It can be customized to the kinds of movements you want this arm to generate. For instance, the specific individual using this prosthetic can customize the grips for the objects in their daily life which are the ones they care the most about. The muscle sensor -based trigger could be replaced with a speech trigger, if so desired.

Summary

AI is empowering a new generation of developers to explore all sorts of novel ideas and mashups. Through his talk on “Connected Arms”, Joseph shows us how the future of prosthetic devices can be transformed by the power of the cloud and AI. Imagine a world in which all future assistive devices are empowered with AI in this fashion. Devices would adapt to individuals, rather than the other way around. Assistive devices will become more affordable, intelligent, cloud-powered and personalized.

Cloud AI is letting us build unexpected things that we would scarcely have imagined.

Such as like an arm that can see.

The AI / ML Blog Team

‘Defining moment’ for Seattle as Microsoft, Amazon, others support Special Olympics USA Games

Michel Kozoris raises his arms after competing in the Special Olympics World Summer Games in 2011. (Photo via Special Olympics)

Seattle-area tech companies are making a statement about building an inclusive workforce as the city hosts one of the largest sporting events in the region’s history.

More than 3,000 athletes will arrive in the Emerald City next week for the 2018 Special Olympics USA Games and compete across 14 sports. The event, now in its 50th year, not only spotlights the talent of athletes with intellectual disabilities, but perhaps more importantly promotes inclusion beyond the playing field.

“In many ways, this can and should be a defining moment for the Seattle community,” Microsoft President Brad Smith said in an interview with GeekWire. “It’s a great opportunity to show the nation that we aspire to be a real city and region of inclusion.”

This group raised a Special Olympics flag atop the Space Needle in Seattle on Friday morning. From left to right: Beth Knox, CEO, Special Olympics USA Games; Allen Stone,singer/songwriter; Devon Adelman, Special Olympics athlete; Brad Smith, Microsoft President; and Taye Diggs, singer and author.

Microsoft is the presenting sponsor of the games. The Redmond, Wash.-based tech giant is a long-time supporter of the Special Olympics through donations of software, technical assistance and even cell phones over the years.

But the company is stepping up even more with the games in its backyard this year. It’s a reflection of Microsoft’s broader focus on accessibility — one that got away from the company until Satya Nadella took over as CEO in 2014, Smith said. “We lost our industry leadership position in terms of meeting the needs of people with disabilities,” he said.

That renewed commitment has helped spur a larger culture shift across the company, Smith said. He pointed to products like the new Xbox Adaptive Controller and the company’s $25 million AI for Accessibility Initiative as examples.

More than 2,000 employees will be volunteering at the games next week. Microsoft’s Xbox division is also hosting the first-ever video game tournament at the USA Games.

“If you think about our mission of empowering everyone, then fundamentally we have to do a good job of meeting the needs of the billion-plus people on the planet that have some kind of disability, temporary or permanent,” Smith added. He and Nadella will speak at the Opening Ceremony on Sunday at Husky Stadium with Washington Gov. Jay Inslee.

Devon Adelman, a Special Olympics USA Games athlete and ambassador, speaks at Microsoft’s Ability Summit in May. (Photo via Microsoft)

Microsoft, Amazon, Starbucks, and other companies are participating in the first-ever job fair at a Special Olympics USA Games this year called “Journey of Employment,” where athletes will gain career advice and meet with potential employers. It’s designed to raise awareness for a talent pool that organizers say is often overlooked.

“These partners are committed to creating inclusive work cultures and to helping improve the unemployment rate for people with disabilities,” said Special Olympics USA Games CEO Beth Knox.

The unemployment rate of people with disabilities was 9.2 percent in 2017, double that of the rate for those without disabilities, according to the Bureau of Labor Statistics. And even with government support, nearly 29 percent of disabled, working-age Americans live below the poverty level, according to U.S. Census data. That line is set at $12,082 for one person.

Artists known as “They Drift” created a mural that was placed on Amazon’s campus in Seattle as part of the “Choose to Include” campaign for the 2018 Special Olympics USA Games. (Photo via Amazon)

Amazon will also be heavily involved next week. The e-commerce giant is sponsoring the closing ceremony near its campus in South Lake Union and will have more than 400 employees volunteering throughout the games. It is also donating boxed lunches for athletes throughout the week via FareStart and launched a neighborhood restaurant campaign this month called “Choose to Include.”

Amazon says it is committed to inclusive hiring. It runs an Alternative Workforce Supplier Program that identifies people with disabilities for hire in the company’s fulfillment centers. The company also partners with Northwest Center, a Seattle-based nonprofit supporting mentally and physically disabled adults and children that provides employees to Amazon. Many of its products and software services offer accessibility features, too.

“At Amazon, diversity and inclusion are an inherent part of our culture,” said Beth Galetti, Amazon’s senior vice president of human resources. “The unique talents, experiences, and backgrounds of our employees are the driving force which enables us to build and innovate on behalf of millions of customers around the world.”

Amazon’s Spheres glow purple in Seattle this past March as the company joined others in marking 100 days until the Special Olympics USA Games begin in the city. (Amazon Photo / Jordan Stead)

Bellevue, Wash.-based wireless carrier T-Mobile and coffee giant Starbucks are getting involved as well. T-Mobile, which has more than 7,000 members in its Access for Disabilities Network, created a text-to-give program that enables donations for the games.

Starbucks has hundreds of employees who are volunteering at the event. For the past three years, Starbucks has received a 100 percent score on the Disability Equality Index survey sponsored by the American Association of People with Disabilities and the U.S. Business Leadership Network.

“Going forward, I would like in my lifetime to see where you hire somebody – a barista who has autism and it goes viral and it’s on the “Ellen” show – that that doesn’t get the attention anymore. That it just is commonplace,” Starbucks manager of Equal Opportunity Initiatives Marthalee Galeota said in 2016. “We have one world – one accessible world where disability is adaptability, it’s humanity, it’s innovation and it’s our global responsibility.”

Smith, Microsoft’s president, said the tech industry should consider how products take into account the needs of people with disabilities, while pursuing new breakthroughs that meet their needs in new ways.

He also said employers should involve people with disabilities in the creation of products. One phrase used in the community is “nothing should be created for us, without us.”

“The key to long-term product improvement for this important community is actually to do a much better job of bringing them into our workforce and making them a key part of our overall ecosystem,” Smith said.

You can get tickets to the USA Games or sign up to volunteer here.