Tag Archives: deployments

New Oracle Enterprise Manager release advances hybrid cloud

In a bid to meet customers’ needs for hybrid cloud deployments, Oracle has injected its Oracle Enterprise Manager system with new capabilities to ease cloud migration and hybrid cloud database management.

The software giant unveiled the new Oracle Enterprise Manager release 13.4 on Wednesday, with general availability expected by the end of the first quarter.

The release includes new analytics features for users to make the most of a single database and optimize performance. Lifecycle automation for databases gets a boost in the new release. The update also provides users with new tools to enable enterprises to migrate from an on-premises database to one in the cloud.

“Managing across hybrid on-prem and public cloud resources can be challenging in terms of planning and executing database migrations,” said Mary Johnston Turner, research vice president for cloud management at IDC. “The new Migration Workbench addresses this need by providing customers with guided support for updating and modernizing across platforms, as appropriate for the customer’s specific requirements.”

Beyond helping with migration, Turner noted that Oracle Enterprise Manager 13.4 supports customer choice by enabling consistent management across Oracle Cloud and traditional on-premises resources, which is a recognition that most enterprises are adopting multi-cloud architectures.

The other key addition in Oracle Enterprise Manager 13.4 is advanced machine learning analytics, Turner noted.

“Prior to this release the analytics capabilities were mostly limited to Oracle Management Cloud SaaS [software as a service] solutions, so adding this capability to Enterprise Manager is significant,” she said.

Oracle Enterprise Manager 13.4 features

Nearly all large Oracle customers use Enterprise Manager already, said Mughees Minhas, vice president of product management at Oracle. He said Oracle doesn’t want to force a new management tool on customers that choose to adopt the cloud, which is why the vendor is increasingly integrating cloud management features with Oracle Enterprise Manager.

Managing across hybrid on-prem and public cloud resources can be challenging in terms of planning and executing database migrations.
Mary Johnston TurnerResearch vice president for cloud management, IDC

As users decide to move data from on-premises deployments to the cloud, it’s rarely just an exercise in moving an application from one environment to another without stopping to redesign the workflow, Minhas said.

The migration tool in the new enterprise manager update includes a SQL performance analyzer feature to ensure that database operations are optimized as they move to the cloud. The tool also includes a compatibility checker to verify that on-premises database applications are compatible with the autonomous versions of Oracle database that runs in the cloud.

Migrating to new databases with Enterprise Manager 13.4

Helping organizations migrate to new database versions is one of the key capabilities of the latest version of Oracle Enterprise Manager.

“Normally, you would create a separate test system on-prem where you would install it and then once you’re done with the testing, then you’d upgrade the actual system,” Minhas said. “So we are promoting these use cases to Enterprise Manager through the use of real application testing tools, where we let you create a new database in the cloud to test.”

Intelligent analytics

The new Oracle Enterprise Manager release also benefits from Exadata Warehouse technology, which now enables analytics for Oracle database workloads.

“The goal of a great admin or cloud DBA [database administrator] is that they want to avoid problems before they happen, and not afterwards,” Minhas said. “So we are building analytical capabilities and some algorithms, so they can do some forecasting, so they know limits and are able to take action.”

Minhas said hybrid management will continue to be Oracle’s focus for Oracle Enterprise Manager.

“Over time, you’ll see us doing more use cases where we also let you do the same thing you’re doing on premises in the cloud, using the same APIs users are already familiar with,” Minhas said.

Go to Original Article
Author:

Container backup grows, following container adoption

The popularity of container deployments is reaching a tipping point where all backup vendors will eventually need to be able to support them, industry experts said.

As container technology adoption increases, the need for container backup grows. Until now, most containers have been stateless and required no backup.

“We’re going to be seeing more stateful containers, buoyed by the fact that now there’s ways to protect them,” said Steven Hill, senior analyst at 451 Research.

Tom Barton, CEO of container storage vendor Diamanti, said he is seeing more customers’ containers with persistent storage. Barton said when containers replace virtual machines, they require the same data protection and disaster recovery (DR) requirements.

“I think containers will generally displace VMs in the long-run,” Barton said.

Diamanti recently launched the beta version of its Spektra platform, a Kubernetes management layer designed for migrating Kubernetes workloads between on premises and cloud. Spektra enables high availability and DR for Kubernetes workloads, and Barton said Diamanti and its competitors partner with data protection vendors to provide container backup.

Other products that offer container backup include Veritas NetBackup, which introduced its Docker container support at the beginning of this year, and IBM Spectrum Protect, which has recently entered this space by rolling out snapshotting for Kubernetes users.

Hill shared similar beliefs about containers replacing VMs but stressed it will not be a one-for-one replacement. He said economics will always play a role. He said some applications and workloads will remain that make sense to keep on VMs while others will belong on containers. The situation will vary between organizations, but it won’t be fair to say containers are strictly better than VMs, or vice versa.

Screenshot of vProtect version 3.9
Storware vProtect supports a wide variety of hypervisors.

“You never do everything with just the one tool,” Hill said.

Hill also stressed that containers themselves aren’t a mature market or technology yet, and vendors are still waiting to see how organizations are using them. Customers putting mission-critical applications on containers have nudged demand for data protection, backup, recovery, availability and failover — the same kind of capabilities expected in any environment. Vendors are responding to this demand, but the tools aren’t ready yet.

“Protecting stateful containers is still relatively new. The numbers aren’t there to define a real market,” Hill said.

Marc Staimer, president of Dragon Slayer Consulting, said containers still lack the security, flexibility and resilience of VMs. He chalks that up to containers’ lack of maturity. As customers put containers into production, they will realize the technology’s shortcomings, and vendors will develop products and features to address those problems. Staimer said the industry has recently reached a tipping point where there’s enough container adoption to catch vendor interest.

Staimer acknowledged that when containers mature to the same point where hypervisors are now, there will be widespread replacement. Like Hill, he does not expect it to be a wholesale replacement.

“We like to believe these things are winner-takes-all, but they’re not,” Staimer said. “In tech, nothing goes away.”

Staimer said from a technical standpoint, container backup has unique problems that differentiate it from traditional server, VM and SaaS application backup. The core problem is that containers don’t have APIs to allow for backup software to take a snapshot of the state of the container. Most backup vendors install agents in containers to scan and capture what it needs to build a recoverable snapshot. This takes time and resources, which goes against the intent of containers being lightweight VMs.

Trilio CEO David Safaii said installing agents in containers also create extra hassle for developers because they have to go through an IT admin to conduct their backups. He said there’s a “civil war” between IT managers and DevOps. IT managers need to worry about data protection, security and compliance. These are all important and necessary measures, but they can get in the way of DevOps’s philosophy of continuous and agile development.

Trilio recently launched the beta program for its TrilioVault for Kubernetes, which is an agentless container backup offering. Asigra similarly performs container backup without using agents, as does Poland-based Storware’s vProtect.

Storware vProtect started in the container backup space by focusing on open platforms first, protecting Red Hat OpenShift and Kubernetes projects. Storware CTO Paweł Mączka said no one asked for container data protection in the early days because container workloads were microservices and applications.

Mączka saw customers now use containers as they would VMs. DevOps now put databases in containers, shifting them from stateless to stateful. However, Mączka doesn’t see containers taking over and proliferating to the same point as hypervisors such as VMware vSphere and Microsoft Hyper-V, which vProtect only started supporting in its latest version 3.9 update.
“I don’t think they’ll rule the world, but it’s important to have the [container backup] feature,” Mączka said.

Go to Original Article
Author:

Kubernetes security opens a new frontier: multi-tenancy

SAN DIEGO — As enterprises expand production container deployments, a new Kubernetes security challenge has emerged: multi-tenancy.

Among the many challenges with multi-tenancy in general is that it is not easy to define, and few IT pros agree on a single definition or architectural approach. Broadly speaking, however, multi-tenancy occurs when multiple projects, teams or tenants, share a centralized IT infrastructure, but remain logically isolated from one another.

Kubernetes multi-tenancy also adds multilayered complexity to an already complex Kubernetes security picture, and demands that IT pros wire together a stack of third-party and, at times, homegrown tools on top of the core Kubernetes framework.

This is because core upstream Kubernetes security features are limited to service accounts for operations such as role-based access control — the platform expects authentication and authorization data to come from an external source. Kubernetes namespaces also don’t offer especially granular or layered isolation by default. Typically, each namespace corresponds to one tenant, whether that tenant is defined as an application, a project or a service.

“To build logical isolation, you have to add a bunch of components on top of Kubernetes,” said Karl Isenberg, tech lead manager at Cruise Automation, a self-driving car service in San Francisco, in a presentation about Kubernetes multi-tenancy here at KubeCon + CloudNativeCon North America 2019 this week. “Once you have Kubernetes, Kubernetes alone is not enough.”

Karl Isenberg, Cruise Automation
Karl Isenberg, tech lead manager at Cruise Automation, presents at KubeCon about multi-tenant Kubernetes security.

However, Isenberg and other presenters here said Kubernetes multi-tenancy can have significant advantages if done right. Cruise, for example, runs very large Kubernetes clusters, with up to 1,000 nodes, shared by thousands of employees, teams, projects and some customers. Kubernetes multi-tenancy means more highly efficient clusters and cost savings on data center hardware and cloud infrastructure.

“Lower operational costs is another [advantage] — if you’re starting up a platform operations team with five people, you may not be able to manage five [separate] clusters,” Isenberg added. “We [also] wanted to make our investments in focused areas, so that they applied to as many tenants as possible.”

Multi-tenant Kubernetes security an ad hoc practice for now

The good news for enterprises that want to achieve Kubernetes multi-tenancy securely is that there are a plethora of third-party tools they can use to do it, some of which are sold by vendors, and others open sourced by firms with Kubernetes development experience, including Cruise and Yahoo Media.

Duke Energy Corporation, for example, has a 60-node Kubernetes cluster in production that’s stretched across three on-premises data centers and shared by 100 web applications so far. The platform is comprised of several vendors’ products, from Diamanti hyper-converged infrastructure to Aqua Security Software’s container firewall, which logically isolates tenants from one another at a granular level that accounts for the ephemeral nature of container infrastructure.

“We don’t want production to talk to anyone [outside of it],” said Ritu Sharma, senior IT architect at the energy holding company in Charlotte, N.C., in a presentation at KubeSec Enterprise Summit, an event co-located with KubeCon this week. “That was the first question that came to mind — how to manage cybersecurity when containers can connect service-to-service within a cluster.”

Some Kubernetes multi-tenancy early adopters also lean on cloud service providers such as Google Kubernetes Engine (GKE) to take on parts of the Kubernetes security burden. GKE can encrypt secrets in the etcd data store, which became available in Kubernetes 1.13, but isn’t enabled by default, according to a KubeSec presentation by Mike Ruth, one of Cruise’s staff security engineers.

Google also offers Workload Identity, which matches up GCP identity and access management with Kubernetes service accounts so that users don’t have to manage Kubernetes secrets or Google Cloud IAM service account keys themselves. Kubernetes SIG-Auth looks to modernize how Kubernetes security tokens are handled by default upstream to smooth Kubernetes secrets management across all clouds, but has run into snags with the migration process.

In the meantime, Verizon’s Yahoo Media has donated a project called Athenz to open source, which handles multiple aspects of authentication and authorization in its on-premises Kubernetes environments, including automatic secrets rotation, expiration and limited-audience policies for intracluster communication similar to those offered by GKE’s Workload Identity. Cruise also created a similar open source tool called RBACSync, along with Daytona, a tool that fetches secrets from HashiCorp Vault, which Cruise uses to store secrets instead of in etcd, and injects them into running applications, and k-rail for workload policy enforcement.

Kubernetes Multi-Tenancy Working Group explores standards

While early adopters have plowed ahead with an amalgamation of third-party and homegrown tools, some users in highly regulated environments look to upstream Kubernetes projects to flesh out more standardized Kubernetes multi-tenancy options.

For example, investment banking company HSBC can use Google’s Anthos Config Management (ACM) to create hierarchical, or nested, namespaces, which make for more highly granular access control mechanisms in a multi-tenant environment, and simplifies their management by automatically propagating shared policies between them. However, the company is following the work of a Kubernetes Multi-Tenancy Working Group established in early 2018 in the hopes it will introduce free open source utilities compatible with multiple public clouds.

Sanjeev Rampal, Kubernetes Multi-Tenancy Working Group
Sanjeev Rampal, co-chair of the Kubernetes Multi-Tenancy Working Group, presents at KubeCon.

“If I want to use ACM in AWS, the Anthos license isn’t cheap,” said Scott Surovich, global container engineering lead at HSBC, in an interview after a presentation here. Anthos also requires VMware server virtualization, and hierarchical namespaces available at the Kubernetes layer could offer Kubernetes multi-tenancy on bare metal, reducing the layers of abstraction and potentially improving performance for HSBC.

Homegrown tools for multi-tenant Kubernetes security won’t fly in HSBC’s highly regulated environment, either, Surovich said.

“I need to prove I have escalation options for support,” he said. “Saying, ‘I wrote that’ isn’t acceptable.”

So far, the working group has two incubation projects that create custom resource definitions — essentially, plugins — that support hierarchical namespaces and virtual clusters that create self-service Kubernetes API Servers for each tenant. The working group has also created working definitions of the types of multi-tenancy and begun to define a set of reference architectures.

The working group is also considering certification of multi-tenant Kubernetes security and management tools, as well as benchmark testing and evaluation of such tools, said Sanjeev Rampal, a Cisco principal engineer and co-chair of the group.

Go to Original Article
Author:

Big data systems up ante on data quality measures for users

NEW YORK — In the rush to capitalize on deployments of big data platforms, organizations shouldn’t neglect data quality measures needed to ensure the information used in analytics applications is clean and trustworthy, experienced IT managers said at the 2017 Strata Data Conference here last week.

Several speakers pointed to data quality as a big challenge in their big data environments — one that required new processes and tools to help their teams get a handle on quality issues, as both the volumes of data being fed into corporate data lakes and use of the info by data scientists and other analysts grow.

“The more of the data you produce is used, the more important it becomes, and the more important data quality becomes,” said Michelle Ufford, manager of core innovation for data engineering and analytics at Netflix Inc. “But it’s very, very difficult to do it well — and when you do it well, it takes a lot of time.”

Over the past 12 months, Ufford’s team worked to streamline the Los Gatos, Calif., company’s data quality measures as part of a broader effort to boost data engineering efficiency based on a “simplify and automate” mantra, she said during a Strata session.

A starting point for the data-quality-upgrade effort was “acknowledging that not all data sets are created equal,” she noted. In general, ones with high levels of usage get more data quality checks than lightly used ones do, according to Ufford, but trying to stay on top of that “puts a lot of cognitive overhead on data engineers.” In addition, it’s hard to spot problems just by looking at the metadata and data-profiling statistics that Netflix captures in an internal data catalog, she said.

Calling for help on data quality

To ease those burdens, Netflix developed a custom data quality tool, called Quinto, and a Python library, called Jumpstarter, which are used together to generate recommendations on quality coverage and to set automated rules for assessing data sets. When data engineers run Spark-based extract, transform and load (ETL) jobs to pull in data on use of the company’s streaming media service for analysis, transient object tables are created in separate partitions from the production tables, Ufford said. Calls are then made from the temporary tables to Quinto to do quality checks before the ETL process is completed.

In the future, Netflix plans to expand the statistics it tracks when profiling data and implement more robust anomaly detection capabilities that can better pinpoint “what is problematic or wrong” in data sets, Ufford added. The ultimate goal, she said, is making sure data engineering isn’t a bottleneck for the analytics work done by Netflix’s BI and data science teams and its business units.

2017 Strata Data Conference in New York
Data quality in big data systems was among the topics discussed at the 2017 Strata Data Conference in New York.

Improving data consistency was one of the goals of a cloud-based data lake deployment at Financial Industry Regulatory Authority Inc., an organization in Washington, D.C., that creates and enforces rules for financial markets. Before the big data platform was set up, fragmented data sets in siloed systems made it hard for data scientists and analysts to do their jobs effectively, said John Hitchingham, director of performance engineering at the not-for-profit regulator, more commonly known as FINRA.

A homegrown data catalog, called herd, was “a real key piece for making this all work,” Hitchingham said in a presentation at the conference. FINRA collects metadata and data lineage info in the catalog; it also lists processing jobs and related data sets there, and it uses the catalog to track schemas and different versions of data in the big data architecture, which runs in the Amazon Web Services (AWS) cloud.

To help ensure the data is clean and consistent, Hitchingham’s team runs validation routines after it’s ingested into Amazon Simple Storage Service (S3) and registered in the catalog. The validated data is then written back to S3, completing a process that he said also reduces the amount of ETL processing required to normalize and enrich data sets before they’re made available for analysis.

Data quality takes a business turn

Brendan Aldrich, CDO at Ivy Tech Community CollegeBrendan Aldrich

The analytics team at Ivy Tech Community College in Indianapolis also does validation checks as data is ingested into its AWS-based big data system — but only to make sure the data matches what’s in the source systems from which it’s coming. The bulk of the school’s data quality measures are now carried out by individual departments in their own systems, said Brendan Aldrich, Ivy Tech’s chief data officer.

“Data cleansing is a never-ending process,” Aldrich said in an interview before speaking at the conference. “Our goal was, rather than getting on that treadmill, why not engage users and get them involved in cleansing the data where it should be done, in the front-end systems?”

That process started taking shape when Ivy Tech, which operates 45 campuses and satellite locations across Indiana, deployed the cloud platform and Hitachi Vantara’s Pentaho BI software several years ago to give its business users self-service analytics capabilities. And it was cemented in July 2016 when the college hired a new president who mandated that business decisions be based on data, Aldrich said.

The central role data plays in decision-making gives departments a big incentive to ensure information is accurate before it goes into the analytics system, he added. As a result, data quality problems are being found and fixed more quickly now, according to Aldrich. “Even if you’re cleansing data centrally, you usually don’t find [an issue] until someone notices it and points it out,” he said. “In this case, we’re cleansing it faster than we were before.”