Tag Archives: source

AWS and Microsoft announce Gluon, making deep learning accessible to all developers – News Center

New open source deep learning interface allows developers to more easily and quickly build machine learning models without compromising training performance. Jointly developed reference specification makes it possible for Gluon to work with any deep learning engine; support for Apache MXNet available today and support for Microsoft Cognitive Toolkit coming soon.

SEATTLE and REDMOND, Wash. — Oct. 12, 2017 — On Thursday, Amazon Web Services Inc. (AWS), an Amazon.com company (NASDAQ: AMZN), and Microsoft Corp. (NASDAQ: MSFT) announced a new deep learning library, called Gluon, that allows developers of all skill levels to prototype, build, train and deploy sophisticated machine learning models for the cloud, devices at the edge and mobile apps. The Gluon interface currently works with Apache MXNet and will support Microsoft Cognitive Toolkit (CNTK) in an upcoming release. With the Gluon interface, developers can build machine learning models using a simple Python API and a range of prebuilt, optimized neural network components. This makes it easier for developers of all skill levels to build neural networks using simple, concise code, without sacrificing performance. AWS and Microsoft published Gluon’s reference specification so other deep learning engines can be integrated with the interface. To get started with the Gluon interface, visit https://github.com/gluon-api/gluon-api/.

Developers build neural networks using three components: training data, a model and an algorithm. The algorithm trains the model to understand patterns in the data. Because the volume of data is large and the models and algorithms are complex, training a model often takes days or even weeks. Deep learning engines like Apache MXNet, Microsoft Cognitive Toolkit and TensorFlow have emerged to help optimize and speed the training process. However, these engines require developers to define the models and algorithms up front using lengthy, complex code that is difficult to change. Other deep learning tools make model-building easier, but this simplicity can come at the cost of slower training performance.

The Gluon interface gives developers the best of both worlds — a concise, easy-to-understand programming interface that enables developers to quickly prototype and experiment with neural network models, and a training method that has minimal impact on the speed of the underlying engine. Developers can use the Gluon interface to create neural networks on the fly, and to change their size and shape dynamically. In addition, because the Gluon interface brings together the training algorithm and the neural network model, developers can perform model training one step at a time. This means it is much easier to debug, update and reuse neural networks.

“The potential of machine learning can only be realized if it is accessible to all developers. Today’s reality is that building and training machine learning models require a great deal of heavy lifting and specialized expertise,” said Swami Sivasubramanian, VP of Amazon AI. “We created the Gluon interface so building neural networks and training models can be as easy as building an app. We look forward to our collaboration with Microsoft on continuing to evolve the Gluon interface for developers interested in making machine learning easier to use.”

“We believe it is important for the industry to work together and pool resources to build technology that benefits the broader community,” said Eric Boyd, corporate vice president of Microsoft AI and Research. “This is why Microsoft has collaborated with AWS to create the Gluon interface and enable an open AI ecosystem where developers have freedom of choice. Machine learning has the ability to transform the way we work, interact and communicate. To make this happen we need to put the right tools in the right hands, and the Gluon interface is a step in this direction.”

“FINRA is using deep learning tools to process the vast amount of data we collect in our data lake,” said Saman Michael Far, senior vice president and CTO, FINRA. “We are excited about the new Gluon interface, which makes it easier to leverage the capabilities of Apache MXNet, an open source framework that aligns with FINRA’s strategy of embracing open source and cloud for machine learning on big data.”

“I rarely see software engineering abstraction principles and numerical machine learning playing well together — and something that may look good in a tutorial could be hundreds of lines of code,” said Andrew Moore, dean of the School of Computer Science at Carnegie Mellon University. “I really appreciate how the Gluon interface is able to keep the code complexity at the same level as the concept; it’s a welcome addition to the machine learning community.”

“The Gluon interface solves the age old problem of having to choose between ease of use and performance, and I know it will resonate with my students,” said Nikolaos Vasiloglou, adjunct professor of Electrical Engineering and Computer Science at Georgia Institute of Technology. “The Gluon interface dramatically accelerates the pace at which students can pick up, apply and innovate on new applications of machine learning. The documentation is great, and I’m looking forward to teaching it as part of my computer science course and in seminars that focus on teaching cutting-edge machine learning concepts across different cities in the U.S.”

“We think the Gluon interface will be an important addition to our machine learning toolkit because it makes it easy to prototype machine learning models,” said Takero Ibuki, senior research engineer at DOCOMO Innovations. “The efficiency and flexibility this interface provides will enable our teams to be more agile and experiment in ways that would have required a prohibitive time investment in the past.”

The Gluon interface is open source and available today in Apache MXNet 0.11, with support for CNTK in an upcoming release. Developers can learn how to get started using Gluon with MXNet by viewing tutorials for both beginners and experts available by visiting https://mxnet.incubator.apache.org/gluon/.

About Amazon Web Services

For 11 years, Amazon Web Services has been the world’s most comprehensive and broadly adopted cloud platform. AWS offers over 90 fully featured services for compute, storage, networking, database, analytics, application services, deployment, management, developer, mobile, Internet of Things (IoT), Artificial Intelligence (AI), security, hybrid and enterprise applications, from 44 Availability Zones (AZs) across 16 geographic regions in the U.S., Australia, Brazil, Canada, China, Germany, India, Ireland, Japan, Korea, Singapore, and the UK. AWS services are trusted by millions of active customers around the world — including the fastest-growing startups, largest enterprises, and leading government agencies — to power their infrastructure, make them more agile, and lower costs. To learn more about AWS, visit https://aws.amazon.com.

About Amazon

Amazon is guided by four principles: customer obsession rather than competitor focus, passion for invention, commitment to operational excellence, and long-term thinking. Customer reviews, 1-Click shopping, personalized recommendations, Prime, Fulfillment by Amazon, AWS, Kindle Direct Publishing, Kindle, Fire tablets, Fire TV, Amazon Echo, and Alexa are some of the products and services pioneered by Amazon. For more information, visit www.amazon.com/about and follow @AmazonNews.

About Microsoft

Microsoft (Nasdaq “MSFT” @microsoft) is the leading platform and productivity company for the mobile-first, cloud-first world, and its mission is to empower every person and every organization on the planet to achieve more.

For more information, press only:

Microsoft Media Relations, WE Communications for Microsoft, (425) 638-7777, rrt@we-worldwide.com

Note to editors: For more information, news and perspectives from Microsoft, please visit the Microsoft News Center at http://news.microsoft.com. Web links, telephone numbers and titles were correct at time of publication, but may have changed. For additional assistance, journalists and analysts may contact Microsoft’s Rapid Response Team or other appropriate contacts listed at http://news.microsoft.com/microsoft-public-relations-contacts.

Windows DevOps shops quickly gain on Linux counterparts

Almost overnight, Windows DevOps has gained ground on the open source world.

Windows shops have a well-earned reputation for conservatism, and a deeply entrenched set of legacy enterprise applications that often hinder automated application development. However, Microsoft products have recently focused on Windows DevOps support. There’s still work to do to underpin Windows container orchestration, but IT pros in Windows DevOps shops are determined to break free of stodgy stereotypes.

Those stereotypes are based in reality. Microsoft shops have been reluctant to deploy early versions of products, and some service providers and consultants that work with Windows-focused enterprises still encounter that history as they contend with DevOps.

In the last three years, Microsoft has lagged behind its open source counterparts in offering DevOps products, particularly for continuous deployment and application release automation, critics said. That lag, plus being locked in to Microsoft tools, is what holds back Windows DevOps.

“Microsoft is making some inroads,” said Brandon Cipes, managing director of DevOps at cPrime, an Agile consulting firm in Foster City, Calif. “They’re finally starting to open up compatibility with other things, but they’re years and years behind the ecosystem that’s developed in open source.”

Third-party tools bridge Windows DevOps gaps

Windows DevOps shops have cobbled together automation pipelines with inefficient multi-hop handoffs between Microsoft apps and third-party tools, Cipes said. For many companies, switching over to a Linux-based stack is easier said than done.

“People get on Microsoft and they never leave,” he said. “We have clients that do a lot of Linux, but everyone has at least one department or one corner of the office that’s still on Microsoft and they will openly comment that they’ll never completely remove themselves from it.”

Tools from vendors such as TeamCity, Octopus Deploy, Electric Cloud and CA’s Automic have helped early adopters. One such firm, Urban Science, a data analysis company that specializes in the automotive industry, uses Electric Cloud’s ElectricFlow continuous integration and continuous delivery (CI/CD) tool to automate software delivery in a heavily Windows-based environment.

“Having the orchestration of ElectricFlow allows us to keep one perspective in mind when we’re creating a workflow,” said Marc Priolo, configuration manager at Urban Science, based in Detroit.

Developers and testers have seen what we’ve done in production and they want the same kind of automation.
Aloisio Rochaoperations specialist, NetEnt

ElectricFlow manages DevOps on Windows for about 80% of the company’s IT environment — “we try to use that as one tool to rule them all,” Priolo said. The other 20% of the work mostly involves handoffs from other tools such as Microsoft’s Team Foundation Server (TFS) to ElectricFlow — and here organizational inertia has held back Urban Science, he said.

“The other 20% would mean that our developers would have to change the way they interact with TFS, and it’s just not been a priority for us to change that,” Priolo said.

Occasionally, cPrime’s Windows clients are left with islands of automation when they must integrate third-party DevOps tools with older versions of Microsoft software, Cipes said.

“If you can’t integrate one little bit of automation, it gets you just such a short bit of the way,” he said. “A lot of people are trying to figure out how to deal with getting past that.”

Windows DevOps shops have succeeded in automating infrastructure with tools such as ElectricFlow. NetEnt, an online gaming systems service provider in Sweden, has rolled out ElectricFlow to manage deployments to its production infrastructure even before it automates the rest of the process.

“We’ve tied in all components that are needed to create servers, deploying and upgrading our applications, to give us some more deployment speed and free us up to find other bottlenecks,” said Aloisio Rocha, operations specialist at NetEnt. “We are looking to shift that left now, since the developers and testers have seen what we’ve done in production and they want the same kind of automation.”

Next, NetEnt will use ElectricFlow’s API integration with VMware virtual machines to automate the creation of and updates to SQL Server databases. Such structured apps are a common DevOps challenge regardless of operating system.

“What we’re using right now is PowerShell scripts, so we have a middle hand from ElectricFlow to VMware’s API,” Rocha said. “We would like to skip those PowerShell scripts and write directly to VMware’s API.”

Microsoft products recast the Windows DevOps equation

For other Windows DevOps shops that struggle with islands of automation, the good news is that the most recent versions of Microsoft software are tightly integrated with third-party tools through REST APIs, and also offer more native features.

This year, Windows DevOps products, such as TFS, have improved support for continuous application deployments to production, and some enterprise IT shops have put them to use.

TFS 2015, for example, fell short in that it didn’t have a release pipeline until update 3, but TFS 2017 changed that, said Anthony Terra III, manager of software architecture and development for a law firm in the Philadelphia area.

“We have a full release pipeline now, and we can set it up so that business analysts are the ones that dictate when things get pushed to production,” Terra said. “We do hands-off deployments, and run three or four production deployments a day if we want to, without any issue.”

DevOps shops in the Azure Cloud also have new options in the latest versions of Visual Studio Team Services (VSTS), a SaaS version of TFS that a majority of VSTS users deploy multiple times a day, said Sam Guckenheimer, product owner for VSTS at Microsoft.

“There has been a lot of work in the most recent releases of Windows to make it leaner for server apps so that you could have a small footprint on your VM for Windows, and containerization is another step in that process,” he said.

Microsoft has added features to VSTS in the last six to 12 months to make it the best tool for CI/CD in Azure’s PaaS and IaaS platforms, Guckenheimer said. It has also shored up a workflow that uses Git for code review, testing and quality assurance, and added support for DevOps techniques such as kanban in VSTS. Further updates will facilitate coordination across teams and higher-level views of development teams’ status and assets.

Beth Pariseau is senior news writer for TechTarget’s Data Center and Virtualization Media Group. Write to her at bpariseau@techtarget.com or follow @PariseauTT on Twitter.