Tag Archives: cloudnative

SaltStack infrastructure-as-code tools seek cloud-native niche

As IT automation evolves to suit the cloud-native era, earlier approaches such as configuration management and infrastructure-as-code tools must also change with the times.

SaltStack infrastructure-as-code tools, along with products from competitors such as Puppet, Chef and Ansible, must accommodate fresh IT trends, from AI to immutable infrastructure. They must also do so quickly to keep up with the rapid pace of innovation in the IT industry, something SaltStack has failed to do in recent years, company officials acknowledge.

“[Our new approach] will be an accelerant that allows us to create [new products] much more quickly than we have in the past, and in a much more maintainable way,” said Salt open source creator Thomas Hatch, who is also CTO of SaltStack, the project’s commercial backer.

That new approach is an overhauled software development process based on the principles of plugin-oriented programming (POP), first introduced to core SaltStack products in 2018. This week, the company also renewed its claim in cloud-native territory with three new open source modules developed using POP that will help it keep pace with rivals and emerging technologies, Hatch said.

The modules are Heist, which creates “dissolvable” infrastructure-as-code execution agents to better serve ephemeral apps; Umbra, which automatically links IT data streams to AI and machine learning services; and Idem, a redesigned data description language based in YAML that simplifies the enforcement of application state.

Salt open source contributors say POP has already sped up the project’s development, where previously they faced long delays between code contributions and production-ready inclusion in the main Salt codebase.

“I’m the largest contributor of Azure-specific code to the Salt open source project, and I committed the bulk of that code at the beginning of 2017,” said Nicholas Hughes, founder and CEO of IT automation consulting firm EITR Technologies in Sykesville, Md., which is also a licensed reseller of SaltStack’s commercial product.  “It was accepted into the developer branch at that point. It just showed up in the stable branch at the beginning of 2019, nearly two years later.”

The new modules, especially Idem, can also be used to modernize Salt, especially its integrations with cloud service providers, Hughes said.

SaltStack plugin-oriented programming
SaltStack rewrote its infrastructure-as-code tools with plugin-oriented programming, instead of a traditional object-oriented method.

SaltStack revs update engine with POP and Idem

SaltStack’s Hatch introduced the POP method three years ago. This approach is a faster, more flexible alternative to the more traditional object-oriented programming method developers previously used to maintain the project, Hatch said.

“Object-oriented programming [puts] functions and data right next to each other, and the result is … a lot of isolated silos of code and data,” he said. “Then you end up building custom scaffolding to get them all to communicate with each other, which means it can become really difficult to extend that code.”

Plugin-oriented programming, by contrast, is based on small modules that can be developed separately and merged quickly into a larger codebase.

The new modules rolled out this week serve as a demonstration of how much more quickly the development of Salt and SaltStack infrastructure-as-code tools can move using POP, Hatch said. While an earlier project, Salt SSH, took one engineer two months to create a minimum viable product, and another six months to polish, Heist took one engineer a week and a half to stand up and another two weeks to refine, he said.

Similar open source projects that maintain infrastructure-as-code tools, such as HashiCorp’s Terraform, had long since broken up their codebases into more modular pieces to speed development, Hughes said. He also contributes Azure integration code to Terraform’s open source community.

[Idem and POP] will allow us to move and iterate and build out [the codebase] much more easily.
Nicholas HughesCEO, EITR Technologies

Now, Hughes said he has high hopes for Idem as a vehicle to further modernize cloud provider integrations in open source Salt, and he has already ported all the Azure code he wrote for Salt into Idem using POP.

“It will allow us to move and iterate and build out those codebases much more easily, and version and handle them separately,” he said. He’d also like to see Salt’s open source AWS integrations updated to work with Idem, as well as Salt functions such as the event bus, which ties in with third-party APIs to orchestrate CI/CD and IT monitoring systems alongside infrastructure.

As the cloud working group captain for the Salt open source project, Hughes said he’s put out a call for the community to port more cloud components into Idem, but that’s still a work in progress

Infrastructure-as-code tools ‘reaching the end of their run?’

In the meantime, the breakneck pace of cloud-native technology development waits for no one, and most of SaltStack’s traditional competitors in infrastructure-as-code tools, such as Puppet, Chef and Ansible, have a head start in the race to reinvent themselves.

Puppet has sought a foothold in CI/CD tools with its Distelli acquisition and moved into agentless IT automation, similar to Ansible’s, with Puppet Bolt. Chef overhauled its Ruby codebase using Rust to create the Chef Habitat project years ahead of SaltStack’s POP, in 2015, and expanded into IT security and compliance with Chef InSpec, which rolled out in version 1.0 in 2016.

SaltStack plans to refocus its business primarily on cloud security automation, which Hatch said accounts for 40 percent of the company’s new sales in 2019. It began that expansion in late 2018, but SaltStack has some potential advantages over Chef InSpec, since it can automate security vulnerability remediation without relying on third-party tools, and the company also beat Red Hat Ansible to the security automation punch, which Ansible began in earnest late last year.

Still, Ansible also has the cachet of its IBM/Red Hat backing and well-known network automation prowess.

HashiCorp’s Terraform has a long lead over Salt’s Idem-based POP modules in cloud provisioning integrations, and the company has hot projects to sustain it in other areas of IT, including cloud security, such as Vault secrets management.

“SaltStack seems to be the slowest to redefine themselves, and they’re the smallest [among their competitors], in my view,” said Jim Mercer, analyst at IDC. “The Umbra plugin that could pull them through into the hot area of AI and machine learning certainly isn’t going to hurt them, but there’s only so much growth left here.” A SaltStack spokesperson expressed disagreement with Mercer’s characterization of the company.

As container orchestration tools such as Kubernetes have risen in popularity, they’ve encroached on the traditional configuration management turf of vendors such as SaltStack, Puppet and Chef, though infrastructure-as-code tools such as Terraform remain widely used to automate cloud infrastructure under Kubernetes and to tie in to GitOps workflows.

Still, the market for infrastructure-as-code tools has also begun to erode, in Mercer’s view, with the growth of function-as-a-service products such as AWS Lambda and serverless container approaches such as AWS Fargate that eliminate infrastructure management below the application container level. Even among shops that still manage infrastructure under Kubernetes, fresh approaches to IT automation have begun to horn in on infrastructure as code’s turf, such as Kubernetes Helm, Kubernetes Operators and KUDO Operators created by D2iQ, formerly Mesosphere.

“These tools had their heyday, but they’re reaching the end of their run,” Mercer said. “They’re still widely used for existing apps, but as new cloud-native apps emerge, they’ll start to go the way of the VCR.”

Go to Original Article
Author:

Kubernetes Helm Tiller is dead, and IT pros rejoice

SAN DIEGO — The death of Kubernetes Helm Tiller in version 3 was the talk of the cloud-native world here at KubeCon + CloudNativeCon North America 2019 this week, as the change promises better security and stability for a utility that underpins several other popular microservices management and GitOps tools.

Kubernetes Helm is a package manager used to deploy apps to the container orchestration platform. It’s widely used to deploy enterprise apps to containers through CI/CD pipelines, including GitOps and progressive delivery tools. It’s also a key component for installing and updating the custom resource definitions (CRDs) that underpin the Istio service mesh in upstream environments.

Helm Tiller was a core component of the software in its initial releases, which used a client-server architecture for which Tiller was the server. Helm Tiller acted as an intermediary between users and the Kubernetes API server, and handled role-based access control (RBAC) and the rendering of Helm charts for deployment to the cluster. With the first stable release of Helm version 3 on Nov. 13, however, Tiller was removed entirely, and Helm version 3 now communicates directly with the Kubernetes API Server.

Such was the antipathy for Helm Tiller among users that when maintainers proclaimed the component’s death from the KubeCon keynote stage here this week, it drew enthusiastic cheers.

“At the first Helm Summit in 2018, there was quite a lot of input from the community, especially around, ‘Can we get rid of Tiller?'” said Martin Hickey, a senior software engineer at IBM and a core maintainer of Helm, in a presentation on Helm version 3 here. “[Now there’s] no more Tiller, and the universe is safe again.”

KubeCon Helm keynote
News of Helm Tiller’s demise from the KubeCon keynote stage this week drew cheers from the audience.

Helm Tiller had security and stability issues

IT pros who used previous versions of Helm charts said the client-server setup between Helm clients and Tiller was buggy and unstable, which made it even more difficult to install already complex tools such as Istio service mesh for upstream users.

“Version 3 offers new consistency in the way it handles CRDs, which had weird dependency issues that we ran into with Istio charts,” said Aaron Christensen, principal software engineer at SPS Commerce, a communications network for supply chain and logistics businesses in Minneapolis. “It doesn’t automatically solve the problem, but if the Istio team makes use of version 3, it could really simplify deployments.”

[Now there’s] no more Tiller, and the universe is safe again.
Martin HickeySenior software engineer, IBM and a core maintainer of Helm

Helm Tiller was designed before Kubernetes had its own RBAC features, but once these were added to the core project, Tiller also became a cause for security concerns among enterprises. From a security perspective, Tiller had cluster-wide access and could potentially be used for privilege escalation attacks if not properly secured.

It was possible to lock down Helm Tiller in version 2 — heavily regulated firms such as Fidelity Investments were able to use it in production with a combination of homegrown tools and GitOps utilities from Weaveworks. But the complexity of that task and Helm Tiller stability problems meant some Kubernetes shops stayed away from Helm altogether until now, which led to other problems with rolling out apps on container clusters.

“Helm would issue false errors to our CI/CD pipelines, and say a deployment failed when it didn’t, or it would time out connecting to the Kubernetes API server, which made the deployment pipeline fail,” said Carlos Traitel, senior DevOps engineer at Primerica, a financial services firm in Duluth, Ga.

Primerica tried to substitute kube-deploy, a different open source utility for Helm, but also ran into management complexity with it. Primerica engineers plan to re-evaluate Helm version 3 as soon as possible. The new version uses a three-way merge process for updates, which compares the desired state with the actual state of the cluster along with the changes users want to apply, and could potentially eliminate many common errors during the Helm chart update process.

Despite its difficulties, Helm version 2 was a crucial element of Kubernetes management, SPS’s Christensen said.

“It worked way more [often] than it didn’t — we wouldn’t go back and use something else,” he said. “It helps keep 20-plus resources consistent across our clusters … and we were also able to implement our own automated rollbacks based on Helm.”

Go to Original Article
Author:

Schlumberger, Chevron and Microsoft announce collaboration to accelerate digital transformation – Stories

Global organizations will work together to accelerate development of cloud-native solutions and deliver actionable data insights for the industry

MONACO September 17, 2019 — Tuesday at the SIS Global Forum 2019, Schlumberger, Chevron and Microsoft. announced the industry’s first three-party collaboration to accelerate creation of innovative petrotechnical and digital technologies.

Data is quickly emerging as one of the most valuable assets to any company yet extracting insights from it is often difficult as information gets trapped in internal silos. As part of the collaboration, the three companies will work together to build Azure-native applications in the DELFI* cognitive E&P environment initially for Chevron, which will enable companies to process, visualize, interpret and ultimately obtain meaningful insights from multiple data sources.

DELFI* is a secure, scalable and open cloud-based environment providing seamless E&P software technology across exploration, development, production and midstream. Chevron and Schlumberger will combine their expertise and resources to accelerate the deployment of DELFI solutions in Azure, with support and guidance from Microsoft. The parties will ensure the software developments meet the latest standards in terms of security, performance, release management, and are compatible with the Open Subsurface Data Universe (OSDU) Data Platform. Building on this open foundation will amplify the capabilities of Chevron’s petrotechnical experts.

The collaboration will be completed in three phases starting with the deployment of the Petrotechnical Suite in the DELFI environment, followed by the development of cloud-native applications on Azure, and the co-innovation of a suite of cognitive computing native capabilities across the E&P value chain tailored to Chevron’s objectives.

Olivier Le Peuch, chief executive officer, Schlumberger, said, “Combining the expertise of these three global enterprises creates vastly improved and digitally enabled petrotechnical workflows. Never before has our industry seen a collaboration of this kind, and of this scale. Working together will accelerate faster innovation with better results, marking the beginning of a new era in our industry that will enable us to elevate performance across our industry’s value chain.”

“There is an enormous opportunity to bring the latest cloud and AI technology to the energy sector and accelerate the industry’s digital transformation,” said Satya Nadella, CEO of Microsoft. “Our partnership with Schlumberger and Chevron delivers on this promise, applying the power of Azure to unlock new AI-driven insights that will help address some of the industry’s—the world’s—most important energy challenges, including sustainability.”

Joseph C. Geagea, executive vice president, technology, projects and services, Chevron, said, “We believe this industry-first advancement will dramatically accelerate the speed with which we can analyze data to generate new exploration opportunities and bring prospects to development more quickly and with more certainty. It will pull vast quantities of information into a single source amplifying our use of artificial intelligence and high-performance computing built on an open data ecosystem.”

About Schlumberger

Schlumberger is the world’s leading provider of technology for reservoir characterization, drilling, production, and processing to the oil and gas industry. With product sales and services in more than 120 countries and employing approximately 100,000 people who represent over 140 nationalities, Schlumberger supplies the industry’s most comprehensive range of products and services, from exploration through production, and integrated pore-to-pipeline solutions that optimize hydrocarbon recovery to deliver reservoir performance.

Schlumberger Limited has executive offices in Paris, Houston, London, and The Hague, and reported revenues of $32.82 billion in 2018. For more information, visit.

About Chevron

Chevron Corporation is one of the world’s leading integrated energy companies. Through its subsidiaries that conduct business worldwide, the company is involved in virtually every facet of the energy industry. Chevron explores for, produces and transports crude oil and natural gas; refines, markets and distributes transportation fuels and lubricants; manufactures and sells petrochemicals and additives; generates power; and develops and deploys technologies that enhance business value in every aspect of the company’s operations. Chevron is based in San Ramon, Calif. More information about Chevron is available at www.chevron.com.

About Microsoft

Microsoft (Nasdaq “MSFT” @microsoft) enables digital transformation for the era of an intelligent cloud and an intelligent edge. Its mission is to empower every person and every organization on the planet to achieve more.

###

*Mark of Schlumberger

For further information, contact:

Moira Duff
Corporate Communication Manager−Western Hemisphere
Schlumberger
Tel: +1 281 285 4376
[email protected]

Sean Comey
Sr. Advisor, External Affairs
Chevron
Tel: +1 925 842 5509
[email protected]

Microsoft Media Relations
WE Communications for Microsoft
(425) 638-7777
[email protected]

Go to Original Article
Author: Microsoft News Center

Building cloud-native applications with Azure and HashiCorp

With each passing year, more and more developers are building cloud-native applications. As developers build more complex applications they are looking to innovators like Microsoft Azure and HashiCorp to reduce the complexity of building and operating these applications. HashiCorp and Azure have worked together on a myriad of innovations. Examples of this innovation include tools that connect cloud-native applications to legacy infrastructure and tools that secure and automate the continuous deployment of customer applications and infrastructure. Azure is deeply committed to being the best platform for open source software developers like HashiCorp to deliver their tools to their customers in an easy-to use, integrated way. Azure innovation like the managed applications platform that power HashiCorp’s Consul Service on Azure are great examples of this commitment to collaboration and a vibrant open source startup ecosystem. We’re also committed to the development of open standards that help these ecosystems move forward and we’re thrilled to have been able to collaborate with HashiCorp on both the CNAB (Cloud Native Application Bundle) and SMI (Service Mesh Interface) specifications.

Last year at HashiConf 2018, I had the opportunity to share how we had started to integrate Terraform and Packer into the Azure platform. I’m incredibly excited to get the opportunity to return this year to share how these integrations are progressing and to share a new collaboration on cloud native networking. With this new work we now have collaborations that help customers connect and operate their applications on Azure using HashiCorp technology.

Connect — HashiCorp Consul Service on Azure

After containers and Kubernetes, one of the most important innovations in microservices has been the development of the concept of a service mesh. Earlier this year we partnered with HashiCorp and others to announce the release of Service Mesh Interface, a collaborative, implementation agnostic API for the configuration and deployment of service mesh technology. We collaborated with HashiCorp to produce a control rules implementation of the traffic access control (TAC) using Consul Connect. Today we’re excited that Azure customers can take advantage of HashiCorp Consul Services on Azure powered by the Azure Managed Applications platform. HashiCorp Consul provides a solution to simplify and secure service networking and with this new managed offering, our joint customers can focus on the value of Consul while confident that the experts at HashiCorp are taking care of the management of the service. Reducing complexity for customers and enabling them to focus on cloud native innovation.

Provision — HashiCorp Terraform on Azure

HashiCorp Terraform is a great tool for doing declarative deployment to Azure. We’re seeing great momentum with adoption of HashiCorp Terraform on Azure as the number of customers has doubled since the beginning of the year – customers are using Terraform to automate Azure infrastructure deployment and operation in a variety of scenarios. 

The momentum is fantastic on the contribution front as well with nearly 180 unique contributors to the Terraform provider for Azure Resource Manager. The involvement from the community with our increased 3-week cadence of releases (currently at version 1.32) ensures more coverage of Azure services by Terraform. Additionally, after customer and community feedback regarding the need for additional Terraform modules for Azure, we’ve been working hard at adding high quality modules and now have doubled the number of Azure modules in the terraform registry, bringing it to over 120 modules. 

We believe all these additional integrations enable customers to manage infrastructure as code more easily and simplify managing their cloud environments. Learn more about Terraform on Azure.

Microsoft and HashiCorp are working together to provide integrated support for Terraform on Azure. Customers using Terraform on Microsoft’s Azure cloud are mutual customers, and both companies are united to provide troubleshooting and support services. This joint entitlement process provides collaborative support across companies and platforms while delivering a seamless customer experience. Customers using Terraform Provider for Azure can file support tickets to Microsoft support. Customers using Terraform on Azure support can file support tickets to Microsoft or HashiCorp.

Deploy — Collaborating on Cloud Native Application Bundles specification

One of the critical problems solved by containers is the hermetic packaging of a binary into a package that is easy to share and deploy around the world. But a cloud-native application is more than a binary, and this is what led to the co-development, with HashiCorp and others, of the Coud Native Application Bundle (CNAB) specification. CNABs  allow you to package images alongside configuration tools like Terraform and other artifacts to allow a user to seamlessly deploy an application from a single package. I’ve been excited to see the community work together to build the specification to a 1.0 release that shows CNAB is ready for all of the world’s deployment needs. Congratulations to the team on the work and the fantastic partnership.

If you want to learn more about the ways in which Azure and HashiCorp collaborate to make cloud-native development easier, please check out the links below:

Go to Original Article
Author: Microsoft News Center