Tag Archives: File

NetApp digs into Talon Storage for Cloud OnTap

The NetApp cloud ecosystem gained some muscle for shared file with the acquisition of software-defined vendor Talon Storage.

NetApp said the Talon Storage Fast software suite is already integrated in NetApp Cloud OnTap as part of an ongoing partnership. Talon provides global file caching and syncs data locally from public clouds to a company’s remote branch offices. The NetApp integration does not include local OnTap clusters.

The NetApp cloud file storage is available with two products. NetApp Cloud Volumes OnTap allows a storage administrator to run NetApp storage in the public cloud in the same manner as on premises. Customers also may run NetApp file storage as a service in Amazon Web Services, Google Cloud Platform and Microsoft Azure.

Those NetApp cloud offerings have direct connectivity and integration with Talon, said Anthony Lye, a NetApp senior vice president and GM of its cloud business unit.

Talon Storage is designed to mitigate the performance penalties that occur when shared cloud is used for local processing. The Talon technology runs as a virtual machine. Ctera Networks, Nasuni and Panzura are Talon’s chief competitors. NetApp did not disclose the purchase price, or how many Talon employees will join NetApp.

Cloud storage works well with cloud-native applications, but performance breaks down when more and more data is consumed locally. Major cloud providers are addressing the issue with products by placing storage closer to on-premises users, including AWS Outposts and Google Anthos.

“Talon is a nice acquisition for NetApp. Small player, but good technology,” said Steve McDowell, a senior analyst for storage and data center technologies at Moor Insights & Strategy, based in Austin, Texas.

Support for Talon Storage customers

Lye said Talon gives NetApp new tools to help customers shift more data workloads to the cloud. He added that it lowers the cost for organizations that need to sync data between multiple locations.

 “The only way you could get low response times historically was to sync the files between all the different remote offices and branch offices. Talon uses memory on the remote machine and has an intelligence to place frequently accessed files into the memory. The files are cached remotely, even though the source of truth still exists on the back-end file server,” Lye said.

NetApp has been exposing functionality to third-party vendors via RESTful APIs, which Lye said helped speed up the Talon integration. He said NetApp has gained more than 1,000 Talon Storage customers. He said NetApp will support Talon customers regardless of their back-end storage.

“We are promoting aggressively to new prospects a bundle that would include Talon Storage and NetApp Cloud Volumes, at a significantly lower TCO,” Lye said.

Talon Storage is the fifth NetApp cloud software acquisition since 2017. The company added compliance software vendor Cognigo last year and Kubernetes specialist Stackpoint.io in 2018, one year after landing memory startup Plexistor and cloud orchestration player GreencloudIQ.

Go to Original Article
Author:

NTFS vs. ReFS – How to Decide Which to Use

By now, you’ve likely heard of Microsoft’s relatively recent file system “ReFS”. Introduced with Windows Server 2012, it seeks to exceed NTFS in stability and scalability. Since we typically store the VHDXs for multiple virtual machines in the same volume, it seems as though it pairs well with ReFS. Unfortunately, it did not… in the beginning. Microsoft has continued to improve ReFS in the intervening years. It has gained several features that distanced it from NTFS. With its maturation, should you start using it for Hyper-V? You have much to consider before making that determination.

What is ReFS?

The moniker “ReFS” means “resilient file system”. It includes built-in features to aid against data corruption. Microsoft’s docs site provides a detailed explanation of ReFS and its features. A brief recap:

  • Integrity streams: ReFS uses checksums to check for file corruption.
  • Automatic repair: When ReFS detects problems in a file, it will automatically enact corrective action.
  • Performance improvements: In a few particular conditions, ReFS provides performance benefits over NTFS.
  • Very large volume and file support: ReFS’s upper limits exceed NTFS’s without incurring the same performance hits.
  • Mirror-accelerated parity: Mirror-accelerated parity uses a lot of raw storage space, but it’s very fast and very resilient.
  • Integration with Storage Spaces: Many of ReFS’s features only work to their fullest in conjunction with Storage Spaces.

Before you get excited about some of the earlier points, I need to emphasize one thing: except for capacity limits, ReFS requires Storage Spaces in order to do its best work.

ReFS Benefits for Hyper-V

ReFS has features that accelerate some virtual machine activities.

  • Block cloning: By my reading, block cloning is essentially a form of de-duplication. But, it doesn’t operate as a file system filter or scanner. It doesn’t passively wait for arbitrary data writes or periodically scan the file system for duplicates. Something must actively invoke it against a specific file. Microsoft specifically indicates that it can greatly speed checkpoint merges.
  • Sparse VDL (valid data length): All file systems record the amount of space allocated to a file. ReFS uses VDL to indicate how much of that file has data. So, when you instruct Hyper-V to create a new fixed VHDX on ReFS, it can create the entire file in about the same amount of time as creating a dynamically-expanding VHDX. It will similarly benefit expansion operations on dynamically-expanding VHDXs.

Take a little bit of time to go over these features. Think through their total applications.

ReFS vs. NTFS for Hyper-V: Technical Comparison

With the general explanation out of the way, now you can make a better assessment of the differences. First, check the comparison tables on Microsoft’s ReFS overview page. For typical Hyper-V deployments, most of the differences mean very little. For instance, you probably don’t need quotas on your Hyper-V storage locations. Let’s make a table of our own, scoped more appropriately for Hyper-V:

  • ReFS wins: Really large storage locations and really large VHDXs
  • ReFS wins: Environments with excessively high incidences of created, checkpointed, or merged VHDXs
  • ReFS wins: Storage Space and Storage Spaces Direct deployments
  • NTFS wins: Single-volume deployments
  • NTFS wins (potentially): Mixed-purpose deployments

I think most of these things speak for themselves. The last two probably need a bit more explanation.

Single-Volume Deployments Require NTFS

In this context, I intend “single-volume deployment” to mean installations where you have Hyper-V (including its management operating system) and all VMs on the same volume. You cannot format a boot volume with ReFS, nor can you place a page file on ReFS. Such an installation also does not allow for Storage Spaces or Storage Spaces Direct, so it would miss out on most of ReFS’s capabilities anyway.

Mixed-Purpose Deployments Might Require NTFS

Some of us have the luck to deploy nothing but virtual machines on dedicated storage locations. Not everyone has that. If your Hyper-V storage volume also hosts files for other purposes, you might need to continue with NTFS. Go over the last table near the bottom of the overview page. It shows the properties that you can only find in NTFS. For standard file sharing scenarios, you lose quotas. You may have legacy applications that require NTFS’s extended properties, or short names. In these situations, only NTFS will do.

Note: If you have any alternative, do not use the same host to run non-Hyper-V roles alongside Hyper-V. Microsoft does not support mixing. Similarly, separate Hyper-V VMs onto volumes apart from volumes that hold other file types.

Unexpected ReFS Behavior

The official content goes to some lengths to describe the benefits of ReFS’s integrity streams. It uses checksums to detect file corruption. If it finds problems, it engages in corrective action. On a Storage Spaces volume that uses protective schemes, it has an opportunity to fix the problem. It does that with the volume online, providing a seamless experience. But, what happens when ReFS can’t correct the problem? That’s where you need to pay real attention.

On the overview page, the documentation uses exceptionally vague wording: “ReFS removes the corrupt data from the namespace”. The integrity streams page does worse: “If the attempt is unsuccessful, ReFS will return an error.” While researching this article, I was told of a more troubling activity: ReFS deletes files that it deems unfixable. The comment section at the bottom of that page includes a corroborating report. If you follow that comment thread through, you’ll find an entry from a Microsoft program manager that states:

ReFS deletes files in two scenarios:

  1. ReFS detects Metadata corruption AND there is no way to fix it. Meaning ReFS is not on a Storage Spaces redundant volume where it can fix the corrupted copy.
  2. ReFS detects data corruption AND Integrity Stream is enabled AND there is no way to fix it. Meaning if Integrity Stream is not enabled, the file will be accessible whether data is corrupted or not. If ReFS is running on a mirrored volume using Storage Spaces, the corrupted copy will be automatically fixed.

The upshot: If ReFS decides that a VHDX has sustained unrecoverable damage, it will delete it. It will not ask, nor will it give you any opportunity to try to salvage what you can. If ReFS isn’t backed by Storage Spaces’s redundancy, then it has no way to perform a repair. So, from one perspective, that makes ReFS on non-Storage Spaces look like a very high risk approach. But…

Mind Your Backups!

You should not overlook the severity of the previous section. However, you should not let it scare you away, either. I certainly understand that you might prefer a partially readable VHDX to a deleted one. To that end, you could simply disable integrity streams on your VMs’ files. I also have another suggestion.

Do not neglect your backups! If ReFS deletes a file, retrieve it from backup. If a VHDX goes corrupt on NTFS, retrieve it from backup. With ReFS, at least you know that you have a problem. With NTFS, problems can lurk much longer. No matter your configuration, the only thing you can depend on to protect your data is a solid backup solution.

When to Choose NTFS for Hyper-V

You now have enough information to make an informed decision. These conditions indicate a good condition for NTFS:

  • Configurations that do not use Storage Spaces, such as single-disk or manufacturer RAID. This alone does not make an airtight point; please read the “Mind Your Backups!” section above.
  • Single-volume systems (your host only has a C: volume)
  • Mixed-purpose systems (please reconfigure to separate roles)
  • Storage on hosts older than 2016 — ReFS was not as mature on previous versions. This alone is not an airtight point.
  • Your backup application vendor does not support ReFS
  • If you’re uncertain about ReFS

As time goes on, NTFS will lose favorability over ReFS in Hyper-V deployments. But, that does not mean that NTFS has reached its end. ReFS has staggeringly higher limits, but very few systems use more than a fraction of what NTFS can offer. ReFS does have impressive resilience features, but NTFS also has self-healing powers and you have access to RAID technologies to defend against data corruption.

Microsoft will continue to develop ReFS. They may eventually position it as NTFS’s successor. As of today, they have not done so. It doesn’t look like they’ll do it tomorrow, either. Do not feel pressured to move to ReFS ahead of your comfort level.

When to Choose ReFS for Hyper-V

Some situations make ReFS the clear choice for storing Hyper-V data:

  • Storage Spaces (and Storage Spaces Direct) environments
  • Extremely large volumes
  • Extremely large VHDXs

You might make an additional performance-based argument for ReFS in an environment with a very high churn of VHDX files. However, do not overestimate the impact of those performance enhancements. The most striking difference appears when you create fixed VHDXs. For all other operations, you need to upgrade your hardware to achieve meaningful improvement.

However, I do not want to gloss over the benefit of ReFS for very large volumes. If you have storage volume of a few terabytes and VHDXs of even a few hundred gigabytes, then ReFS will rarely beat NTFS significantly. When you start thinking in terms of hundreds of terabytes, NTFS will likely show bottlenecks. If you need to push higher, then ReFS becomes your only choice.

ReFS really shines when you combine it with Storage Spaces Direct. Its ability to automatically perform a non-disruptive online repair is truly impressive. On the one hand, the odds of disruptive data corruption on modern systems constitute a statistical anomaly. On the other, no one that has suffered through such an event really cares how unlikely it was.

ReFS vs NTFS on Hyper-V Guest File Systems

All of the above deals only with Hyper-V’s storage of virtual machines. What about ReFS in guest operating systems?

To answer that question, we need to go back to ReFS’s strengths. So far, we’ve only thought about it in terms of Hyper-V. Guests have their own conditions and needs. Let’s start by reviewing Microsoft’s ReFS overview. Specifically the following:

“Microsoft has developed NTFS specifically for general-purpose use with a wide range of configurations and workloads, however for customers specially requiring the availability, resiliency, and/or scale that ReFS provides, Microsoft supports ReFS for use under the following configurations and scenarios…”

I added emphasis on the part that I want you to consider. The sentence itself makes you think that they’ll go on to list some usages, but they only list one: “backup target”. The other items on their list only talk about the storage configuration. So, we need to dig back into the sentence and pull out those three descriptors to help us decide: “availability”, “resiliency”, and “scale”. You can toss out the first two right away — you should not focus on storage availability and resiliency inside a VM. That leaves us with “scale”. So, really big volumes and really big files. Remember, that means hundreds of terabytes and up.

For a more accurate decision, read through the feature comparisons. If any application that you want to use inside a guest needs features only found on NTFS, use NTFS. Personally, I still use NTFS inside guests almost exclusively. ReFS needs Storage Spaces to do its best work, and Storage Spaces does its best work at the physical layer.

Combining ReFS with NTFS across Hyper-V Host and Guests

Keep in mind that the file system inside a guest has no bearing on the host’s file system, and vice versa. As far as Hyper-V knows, VHDXs attached to virtual machines are nothing other than a bundle of data blocks. You can use any combination that works.

Go to Original Article
Author: Eric Siron

New DataCore vFilO software pools NAS, object storage

DataCore Software is expanding beyond block storage virtualization with new file and object storage capabilities for unstructured data.

Customers can use the new DataCore vFilO to pool and centrally manage disparate file servers, NAS appliances and object stores located on premises and in the cloud. They also have the option to install vFilO on top of DataCore’s SANsymphony block virtualization software, now rebranded as DataCore SDS

DataCore CMO Gerardo Dada said customers that used the block storage virtualization asked for similar capabilities on the file side. Bringing diverse systems under central management can give them a consistent way to provision, encrypt, migrate and protect data and to locate and share files. Unifying the data also paves the way for customers to use tools such as predictive analytics, Dada said.

Global namespace

The new vFilO software provides a scale-out file system for unstructured data and virtualization technology to abstract existing storage systems. A global namespace facilitates unified access to local and cloud-based data though standard NFS, SMB, and S3 protocols. On the back end, vFilO communicates with the file systems through parallel NFS to speed response times. The software separates metadata from the data to facilitate keyword queries, the company said.

Users can set policies at volume or more granular file levels to place frequently accessed data on faster storage and less active data on lower cost options. They can control the frequency of snapshots for data protection, and they can archive data in on-premises or public cloud object storage in compressed and deduplicated format to reduce costs, Dada said.

DataCore’s vFilO supports automated load balancing across the diverse filers, and users can add nodes to scale out capacity and performance. The minimum vFilo configuration for high availability is four virtual machines, with one node managing the metadata services and the other handling the data services, Dada said.

DataCore vFilo screenshot
New DataCore vFilO software can pool and manage disparate file servers, NAS appliances and object stores.

File vs. object storage

Steven Hill, a senior analyst of storage technologies at 451 Research, said the industry in general would need to better align file and object storage moving forward to address the emerging unstructured data crisis.

“Most of our applications still depend on file systems, but metadata-rich object is far better suited to longer-term data governance and management — especially in the context of hybrid IT, where much of the data resident in the cloud is based on efficient and reliable objects, ” Hill said.

“File systems are great for helping end users remember what their data is and where they put it, but not very useful for identifying and automating policy-based management downstream,” Hill added. “Object storage provides a highly-scalable storage model that’s cloud-friendly and supports the collection of metadata that can then be used to classify and manage that data over time.”

DataCore expects the primary use cases for vFilO to include consolidating file systems and NAS appliances. Customers can use vFilo to move unused or infrequently accessed files to cheaper cloud object storage to free up primary storage space. They can also replicate files for disaster recovery.

Eric Burgener, a research vice president at IDC, said unstructured data is a high growth area. He predicts vFilO will be most attractive to the company’s existing customers. DataCore claims to have more than 10,000 customers.

“DataCore customers already liked the functionality, and they know how to manage it,” Burgener said. “If [vFilO] starts to get traction because of its ease of use, then we may see more pickup on the new customer side.”

Camberley Bates, a managing director and analyst at Evaluator Group, expects DataCore to focus on the media market and other industries needing high performance.

Pricing for vFilO

Pricing for vFilO is based on capacity consumption, with a 10 TB minimum order. One- and three-year subscriptions are available, with different pricing for active and inactive data. A vFilO installation with 10 TB to 49 TB of active data costs $345 per TB for a one-year subscription and $904 per TB for a three-year subscription. For the same capacity range of inactive data, vFilo would cost $175 per TB for a one-year subscription and $459 per TB for a three-year subscription. DataCore offers volume discounts for customers with higher capacity deployments.

The Linux-based vFilO image can run on a virtual machine or on commodity bare-metal servers. Dada said DataCore recommends separate hardware for the differently architected vFilO and SANsymphony products to avoid resource contention. Both products have plugins for Kubernetes and other container environments, Dada added.

The vFilO software became available this week as a software-only product, but Dada said the company could add an appliance if customers and resellers express enough interest. DataCore launched a hyper-converged infrastructure appliance for SANsymphony over the summer. 

DataCore incorporated technology from partners and open source projects into the new vFilO software, but Dada declined to identify the sources.

Go to Original Article
Author:

How to Create and Manage Hot/Cold Tiered Storage

When I was working in Microsoft’s File Services team around 2010, one of the primary goals of the organization was to commoditize storage and make it more affordable to enterprises. Legacy storage vendors offered expensive products, often consuming a majority of the budget of the IT department and they were slow to make improvements because customers were locked in. Since then, every release of Windows Server has included storage management features which were previously only provided by storage vendors, such as deduplication, replication, and mirroring. These features could be used to manage commodity storage arrays and disks, reducing costs and eliminating vendor lock-in. Windows Server now offers a much-requested feature, the ability to move files between different tiers of “hot” (fast) storage and “cold” (slow) storage.

Managing hot/cold storage is conceptually similar to computer memory cache but at an enterprise scale. Files which are frequently accessed can be optimized to run on the hot storage, such as faster SSDs. Meanwhile, files which are infrequently accessed will be pushed to cold storage, such as older or cheaper disks. These lower priority files will also take advantage of file compression techniques like data deduplication to maximize storage capacity and minimize cost. Identical or varying disk types can be used because the storage is managed as a pool using Windows Server’s storage spaces, so you do not need to worry about managing individual drives. The file placement is controlled by the Resilient File System (ReFS), a file system which is used to optimize and rotate data between the “hot” and “cold” storage tiers in real-time based on their usage. However, using tiered storage is only recommended for workloads that are not regularly accessed. If you have permanently running VMs or you are using all the files on a given disk, there would be little benefit in allocating some of the disk to cold storage. This blog post will review the key components required to deploy tiered storage in your datacenter.

Overview of Resilient File System (ReFS) with Storage Tiering

The Resilient File System was first introduced in Windows Server 2012 with support for limited scenarios, but it has been greatly enhanced through the Windows Server 2019 release. It was designed to be efficient, support multiple workloads, avoid corruption and maximize data availability. More specifically to tiering though, ReFS divides the pool of storage into two tiers automatically, one for high-speed performance and one of maximizing storage capacity. The performance tier receives all the writes on the faster disk for better performance. If those new blocks of data are not frequently accessed, the files will gradually be moved to the capacity tier. Reads will usually happen from the capacity tier, but can also happen from the performance tier as needed.

Storage Spaces Direct and Mirror-Accelerated Parity

Storage Spaces Direct (S2D) is one of Microsoft’s enhancements designed to reduce costs by allowing servers with Direct Attached Storage (DAS) drives to support Windows Server Failover Clustering. Previously, highly-available file server clusters required some type of shared storage on a SAN or used an SMB file share, but S2D allows for small local clusters which can mirror the data between nodes. Check out Altaro’s blog on Storage Spaces Direct for in-depth coverage on this technology.

With Windows Server 2016 and 2019, S2D offers mirror-accelerated parity which is used for tiered storage, but it is generally recommended for backups and less frequently accessed files, rather than heavy production workloads such as VMs. In order to use tiered storage with ReFS, you will use mirror-accelerated parity. This provides decent storage capacity by using both mirroring and a parity drive to help prevent and recover from data loss. In the past, mirroring and parity would conflict and you would usually have to select one of the other.  Mirror-accelerator parity works with ReFS by taking writes and mirroring them (hot storage), then using parity to optimize their storage on disk (cold storage). By switching between these storage optimizations techniques, ReFS provides admins with the best of both worlds.

Creating Hot and Cold Tiered Storage

When configuring hot and cold storage you get to define the ratio of the hot and cold storage. For most workloads, Microsoft recommends allocating 20% to hot and 80% to cold. If you are using high-performance workloads, consider having more hot memory to support more writes. On the flip-side, if you have a lot of archival files, then allocate more cold memory. Remember that with a storage pool you can combine multiple disk types under the same abstracted storage space. The following PowerShell cmdlets show you how to configure a 1,000 GB disk to use 20% (200 GB) for performance (hot storage) and 80% (800 GB) for capacity (cold storage).

Managing Hot and Cold Tiered Storage

If you want to increase the performance of your disk, then you will allocate a great percentage of the disk to the performance (hot) tier. In the following example we use the PowerShell cmdlets to create a 30:70 ratio between the tiers:

Unfortunately, this resizing only changes the ratios of the disks but does not change the size of the partition or volume, so you likely also want to change these using the Resize-Volumes cmdlets.

Optimizing Hot and Cold Storage

Based on the types of workloads you are using, you may wish to further optimize when data is moved between hot and cold storage, which is known as the “aggressiveness” of the rotation. By default, the hot storage will use wait until 85% of its capacity is full before it begins to send data to the cold storage. If you have a lot of write traffic going to the hot storage then you want to reduce this value so that performance-tier data gets pushed to the cold storage quicker. If you have fewer write requests and want to keep data in hot storage longer then you can increase this value. Since this is an advanced configuration option, it must be configured via the registry on every node in the S2D cluster, and it also requires a restart. Here is a sample script to run on each node if you want to change the aggressiveness so that it swaps files when the performance tier reaches 70% capacity:

You can apply this setting cluster-wide by using the following cmdlet:

NOTE: If this is applied to an active cluster, make sure that you reboot one node at a time to maintain service availability.

Wrap-Up

Now you should be fully equipped with the knowledge to optimize your commodity storage using the latest Windows Server storage management features. You can pool your disks with storage spaces, use storage spaces direct (S2D) to eliminate the need for a SAN, and ReFS to optimize the performance and capacity of these drives.  By understanding the tradeoffs between performance and capacity, your organization can significantly save on storage management and hardware costs. Windows Server has made it easy to centralize and optimize your storage so you can reallocate your budget to a new project – or to your wages!

What about you? Have you tried any of the features listed in the article? Have they worked well for you? Have they not worked well? Why or why not? Let us know in the comments section below!


Go to Original Article
Author: Symon Perriman

Google’s Elastifile buy shows need for cloud file storage

Google’s acquisition of startup Elastifile underscored the increasing importance of enterprise-class file storage in the public cloud.

Major cloud providers have long offered block storage for applications that customers run on their compute services and focused on scale-out object storage for the massively growing volumes of colder unstructured data. Now they’re also shoring up file storage as enterprises look to shift more workloads to the cloud.

Google disclosed its intention to purchase Elastifile for an undisclosed sum after collaborating with the startup on a fully managed file storage service that launched early in 2019 on its cloud platform. At the time, Elastifile’s CEO, Erwan Menard, positioned the service as a complement to the Google Cloud Filestore, saying his company’s technology would provide higher performance, scale-out capacity and enterprise-grade features than the Google option.

Integration plans

In a blog post on the acquisition, Google Cloud CEO Thomas Kurian said the teams would join together to integrate the Elastifile technology with Google Cloud Filestore. Kurian wrote that Elastifile’s pioneering software-defined approach would address the challenges of file storage for enterprise-grade applications running at scale in the cloud.

“Google now has the opportunity to create hybrid cloud file services to connect the growing unstructured data at the edge or core data centers to the public cloud for processing,” said Julia Palmer, a vice president at Gartner. She said Google could have needed considerably more time to develop and perfect a scale-out file system if not for the Elastifile acquisition.

Building an enterprise-level, high-performance NFS file system from scratch is “insanely difficult,” said Scott Sinclair, a senior analyst at Enterprise Strategy Group. He said Google had several months to “put Elastifile through its paces,” see that the technology looked good, and opt to buy rather than build the sort of file system that is “essential for the modern application environments that Google wants to sell into.”

Target workloads

Kurian cited examples of companies running SAP and developers building stateful container-based applications that require natively compatible file storage. He noted customers such as Appsbroker, eSilicon and Forbes that use the Elastifile Cloud File Service on Google Cloud Platform (GCP). In the case of eSilicon, the company bursts semiconductor design workflows to Google Cloud when it needs extra compute and storage capacity during peak times, Elastifile has said.

“The combination of Elastifile and Google Cloud will support bringing traditional workloads into GCP faster and simplify the management and scaling of data and compute intensive workloads,” Kurian wrote. “Furthermore, we believe this combination will empower businesses to build industry-specific, high performance applications that need petabyte-scale file storage more quickly and easily.”

Elastifile’s Israel-based engineering team spent four years developing the distributed Elastifile Cloud File System (ECFS). They designed ECFS for hybrid and public cloud use and banked on high-speed flash hardware to prevent metadata server bottlenecks and facilitate consistent performance.

Elastifile emerged from stealth in April 2017, claiming 25 customers, including 16 service providers. Target use cases it cited for ECFS included high-performance NAS, workload consolidation in virtualized environments, big data analytics, relational and NoSQL databases, high-performance computing, and the lift and shift of data and applications to the cloud. Elastifile raised $74 million over four funding rounds, including strategic investments from Dell Technologies, Cisco and Western Digital.

One open question is the degree to which Google will support Elastifile’s existing customers, especially those with hybrid cloud deployments that did not run on GCP. Both Google and Elastifile declined to respond.

Cloud NAS competition

The competitive landscape for the Elastifile Cloud File Service on GCP has included Amazon’s Elastic File System (EFS), Dell EMC’s Isilon on GCP, Microsoft’s Azure NetApp Files, and NetApp on GCP.

“Cloud NAS and cloud file systems are the last mile for cloud storage. Everybody does block. Everybody does object. NAS and file services were kind of an afterthought,” said Henry Baltazar, research director of storage at 451 Research.

But Baltazar said as more companies are thinking about moving their NFS-based legacy applications to the cloud, they don’t want to go through the pain and the cost of rewriting them for object storage or building a virtual file service. He sees Google’s acquisition of Elastifile as “a good sign for customers that more of these services will be available” for cloud NAS.

“Google doesn’t really make infrastructure acquisitions, so it says something that Google would make a deal like this,” Baltazar said. “It just shows that there’s a need.”

Go to Original Article
Author:

For Sale – Dell PowerEdge T20 Server

Dell T20 bought from ServersPlus nearly 3 years ago. Was used as a plex and file server.

All boxed and in Original configuration

CPU is Xeon E3-1225v3
Memory is 4GB EEC​

It’s a heavy beast so would prefer collection, but can arrange shipping with buyer if required.

Looking for £150 +shipping as required

Have a few bits to upgrade with if required: non EEC memory, hard drives, internal SATA card

Price and currency: £150
Delivery: Delivery cost is not included
Payment method: BT / PPG / CoD
Location: Bushey, Herts
Advertised elsewhere?: Advertised elsewhere
Prefer goods collected?: I prefer the goods to be collected

______________________________________________________
This message is automatically inserted in all classifieds forum threads.
By replying to this thread you agree to abide by the trading rules detailed here.
Please be advised, all buyers and sellers should satisfy themselves that the other party is genuine by providing the following via private conversation to each other after negotiations are complete and prior to dispatching goods and making payment:

  • Landline telephone number. Make a call to check out the area code and number are correct, too
  • Name and address including postcode
  • Valid e-mail address

DO NOT proceed with a deal until you are completely satisfied with all details being correct. It’s in your best interest to check out these details yourself.

Panzura tackles multi-cloud data management

Panzura is expanding beyond cloud file services to multi-cloud data management with its new Vizion.ai option, which is designed to enable customers to search, analyze and control data on premises and off premises.

The Campbell, Calif., company’s CEO, Patrick Harr, said the vendor built its Vizion.ai software-as-a-service (SaaS) offering on a new hyperscale multi-cloud data engine orchestrated by Kubernetes. Vizion.ai embeds machine learning and policy functionality for data analytics and control. It features an open API for third-party developers to use the Panzura technology with their own applications, such as internet of things and security monitoring.

Panzura initially focused on helping enterprises shift from legacy file-based NAS systems to object storage in public and private clouds. The vendor sells Freedom NAS filer appliances that cache active data in flash drives for fast access, while shifting colder data to object storage. Users can also run the software in virtual machines (VMs) on their own hardware or on public cloud servers.

With the new Vizion.ai SaaS option, Panzura consolidates and centralizes metadata to facilitate fast indexing in its Freedom NAS products, third-party NAS filers, SaaS applications and public cloud storage. The company integrated open source Elasticsearch technology to enable the distributed search capability.

“We’ve had a lot of requests in the past for how to search data in a multi-cloud fashion. And when I say multi-cloud, I’m not only talking Amazon, Azure and Google. I’m also talking about private cloud,” Harr said.

Visibility into third-party storage

Harr said the Vizion.ai multi-cloud data management service gains visibility into third-party storage through connector technology the company is offering to the open source community. Users download a small VM and plug the software into their Dell EMC, NetApp or Windows filers. The software crawls the NAS systems, takes a snapshot of the metadata and uploads the indices into the Vizion.ai service, Harr said.

Panzura plans to support a private managed option for customers to use the Vizion.ai index, search and analytics capabilities on premises in secure environments, the CEO added. That support is expected by the end of 2018.

Panzura built algorithms for machine learning to examine data access patterns to let the software recommend the most cost-effective storage location, Harr said. Users can look at heat maps of hot, warm and cold data. And they can use the technology for audit purposes, because they can see who has accessed the data at specific times, he said.

The Vizion.ai capabilities extend to restoring data from snapshots and cloning data for test and development. A customer might want to move a select workload’s data to the optimal cloud, such as Google for machine learning, Harr said.

Panzura’s multi-cloud data management effort

Harr said Panzura started designing its new hyperscale multi-cloud data management platform two years ago to be able to service billions of files and objects across multiple clouds. So far, more than 100 customers tested a private beta version of the Vizion.ai service.

Panzura opened its Vizion.ai beta to the public this week. When the service goes live in October, Vizion.ai multi-cloud data management will be priced based on gigabytes of data indexed and managed, Harr said. The company will have a free version for customers to index and search 1 GB of metadata.

Beta tester Prosper Funding, a San Francisco-based peer-to-peer lending company, started using Panzura’s hybrid cloud technology in 2016. Fabian Duarte, a senior storage engineer working out of Prosper’s Phoenix office, said the company deploys Panzura to make content available for collaboration from any data center and for long-term archiving on AWS.

Prosper tested Vizion.ai by uploading streams of content from AWS tiers, where it stores 3 PB of data, Duarte said. Prosper asked Panzura for access to hotter data in the local cache through a URL-enabled link that a user could click to open the file. The system downloads and rehydrates the file into the Panzura platform’s retrieval folder, Duarte said.

The Vizion.ai service looks promising, Duarte said, and Prosper will likely purchase it. He said its index and search could benefit customer service representatives who need to access call logs for training, playback or other purposes. The Vizion.ai service could also assist departments that deal with access log and audit information for compliance and risk management. Duarte said Prosper has been testing the uploading and manipulation of content inside the file system to track usage patterns.

“We’ve already gone through the route of using tools like Active Directory for multifactor authentication,” he said. “But now, to have the visibility to see who’s working on files, moving files, trying to access files allows us a greater level of granularity to bring an additional level of security.”

The usage-tracking info collected by Vizion.ai could show the cost to rehydrate archived content and determine which content is a good candidate to move to cheaper cloud storage, Duarte said.

Panzura Vizion.ai architectural diagram
Where Panzura Vizion.ai fits in.

Hybrid cloud data management

“With Vizion.ai, Panzura has the potential to evolve from the traditional cloud storage gateway use case toward global hybrid cloud data management,” Gartner research director Julia Palmer wrote in an email.

Legacy gateways and other hybrid storage products, until recently, have focused on backup, archiving and tiering data to the cloud, Palmer said. They wrote data in a proprietary format that other vendors’ technology couldn’t use.

Steven Hill, a senior analyst at 451 Research, said Panzura’s traditional competitors include cloud NAS and gateway companies such as Actifio, Ctera Networks, Microsoft’s Avere Systems, Nasuni and SoftNAS, along with Dell EMC, Hewlett Packard Enterprise and IBM.

“Today, there are dozens of vendors in the secondary storage market that are merging file and object as part of a more advanced storage architecture that focuses on the problems of information management, security and protection, rather than providing traditional ‘dumb’ storage,” Hill wrote in an email.

Ctera Networks adds Dell and HPE gateway appliance options

Ctera Networks is aiming to move its file storage services into the enterprise through new partnerships with Dell and Hewlett Packard Enterprise.

The partnerships, launched in June, allow Ctera to bundle its Enterprise File Services Platform on more-powerful servers with greater storage capacity. Ctera previously sold its branded cloud gateway appliances on generic white box hardware at a maximum raw capacity of 32 TB. The new Ctera HC Series Edge Filers include the HC1200 model offering as much as 96 TB, the HC400 with as much as 32 TB and the HC400E at 16 TB on Dell or HPE servers with 3.5-inch SATA HDDs.

The gateway appliances bundle Ctera’s file services that provide users with access to files on premises and transfers colder data to cloud-based, scale-out object storage at the customer site or in public clouds.

The new models include the Petach Tikvah, Israel, company’s first all-flash appliances. The HC1200 is equipped with 3.84 TB SATA SSDs and offers a maximum raw capacity of 46.08 TB. The HC400 tops out at 15.36 TB. The all-flash models use HPE hardware with 2.5-inch read-intensive SSDs that carry a three-year warranty.

Ctera Networks doesn’t sell appliances with a mix of HDDs and SSDs. The HC400 and HC400E are single rack-unit systems with four drive bays, and the HC1200 is a 2U device with 12 drive bays.

“In the past, we had 32 TB of storage, and it would replace a certain size of NAS device. With this one, we can replace a large series of NAS devices with a single device,” Ctera Networks CEO Liran Eshel said.

Ctera HC Series Edge Filers
New Ctera HC Series Edge Filers include Ctera Networks HC1200 (top) and HC400 file storage.

New Ctera Networks appliances enable multiple VMs

The new, more-powerful HC Series Edge Filers will enable customers to run multiple VMware virtual machines (VMs), applications and storage on the same device, Eshel said. The HC Series supports 10 Gigabit Ethernet networking with fiber and copper cabling options.

“Our earlier generation was just a cloud storage gateway. It didn’t do other things,” Eshel said. “With this version, we actually have convergence — multiple applications in the same appliance. Basically, we’re providing top-of-the-line servers with global support.”

The Dell and HPE partnerships will let Ctera Networks offer on-site support within four hours, as opposed to the next-business-day service it provided in the past. Ctera will take the first call, Eshel said, and be responsible for the customer ticket. If it’s a hardware issue, Ctera will dispatch partner-affiliated engineers to address the problem.

Using Dell and HPE servers enables worldwide logistics and support, which is especially helpful for users with global operations.

“It was challenging to do that with white box manufacturing,” Eshel said.

Software-defined storage vendors require these types of partnerships to sell into the enterprise, said Steven Hill, a senior analyst at 451 Research.

“In spite of the increasingly software-based storage model, we find that many customers still prefer to buy their storage as pre-integrated appliances, often based on hardware from their current vendor of choice,” Hill wrote in an e-mail. “This guarantees full hardware compatibility and provides a streamlined path for service and support, as well as compatibility with an existing infrastructure and management platform.”

Cloud object storage options

The Ctera product works with on-premises object storage from Caringo, Cloudian, DataDirect Networks, Dell EMC, Hitachi, HPE, IBM, NetApp, Scality and SwiftStack. It also supports Amazon Web Services, Google Cloud Platform, IBM Cloud, Microsoft Azure, Oracle and Verizon public clouds. Ctera has reseller agreements with HPE and IBM.

Eshel said one multinational customer, WPP, has already rolled out the new appliances in production for use with IBM Cloud.

The list price for the new Ctera HC Series starts at $10,000. Ctera also continues to sell its EC Series appliances on white box hardware. Customers have the option to buy the hardware pre-integrated with the Ctera software or purchase virtual gateway software that they can install on server hypervisors on premises or in Amazon or Azure public clouds.

Quobyte preps 2.0 Data Center File System software update

Quobyte’s updated Data Center File System software adds volume-mirroring capabilities for disaster recovery, support for Mac and Windows clients, and shared access control lists.

The startup, based in Santa Clara, Calif., this week released the 2.0 version of its distributed POSIX-compliant parallel file system to beta testers and expects to make the updated product generally available in January.

The Quobyte software supports file, block and object storage, and it’s designed to scale out IOPS, throughput and capacity linearly on commodity hardware ranging from four to thousands of servers. Policy-based data placement lets users earmark high-performance workloads to flash drives, including faster new NVMe-based PCIe solid-state drives.

Software-defined storage startups face challenges

Despite the additions, industry analysts question whether Quobyte has done enough to stand out in a crowded field of file-system vendors.

Marc Staimer, president of Dragon Slayer Consulting, said Quobyte faces significant hurdles against competition ranging from established giants, such as Dell EMC, to startups, including Elastifile, Qumulo, Rozo Systems, StorOne and WekaIO.

Staimer called features such as shared access control lists (ACLs) and volume mirroring table stakes in the software-defined storage market. He said mirroring — a technology that was hot 20 years ago — protects against hardware failures, but doesn’t go far enough for disaster recovery. He said Quobyte must consider adding versioning and time stamping to protect against software corruption, malware, accidental deletion and problems of that nature.

Steven Hill, a senior storage analyst at 451 Research, said it takes more than features to gain share in the enterprise storage market. He said Quobyte would do well to forge closer hardware partnerships to provide better integration, optimization, support and services.

“Even though software-delivered storage appears to be the trend, many storage customers still seem more interested in the fully supported hardware [and] software appliance model, rather than taking a roll-your-own approach to enterprise storage,  especially when there can be so many different production requirements in play at the same time,” Hill wrote in an email.

Quobyte CEO Bjorn Kolbeck and CTO Felix Hupfeld worked in storage engineering at Google before starting Quobyte in 2013. And Kolbeck claimed the “Google-style operations” that the Quobyte architecture enables would allow users to grow the system and run 24/7 without the need for additional manpower.

According to Kolbeck, fault tolerance is the most important enabler for Google-style operations. He said Quobyte achieves fault tolerance through automated replication, erasure coding, disaster recovery and end-to-end checksums that ensure data integrity. With those capabilities, users can fix broken hardware on their own schedules, he said.

“That’s the key to managing very large installations with hundreds of petabytes with a small team,” Kolbeck said.

Quobyte 2.0

Kolbeck said Quobyte made volume mirroring a priority following requests from commercial customers. The software uses continuous asynchronous replication across geographic regions and clouds to facilitate disaster recovery. Kolbeck said customers would be able replicate the primary site and use erasure coding with remote sites to lower the storage footprint, if they choose.

To expand data sharing across platforms and interfaces, Quobyte 2.0 finalized native drivers for Mac and Windows clients. Its previous version supported Linux, Hadoop and Amazon Simple Storage Service (S3) options for users to read, write and access files.

Kolbeck said adding access control lists will allow users to read and modify them from all interfaces now that Mac and Windows ACLs and S3 permissions map to Quobyte’s internal NFSv4 ACLs.

Quobyte also moved to simplify installation and management through the creation of a cloud-based service to assist with domain name system service configuration. Kolbeck said the company “moved as far away from the command line as possible,” and the system now can walk customers through the installation process.

Kolbeck said Quobyte currently has about 25 customers running the software in production. He said the company targets commercial high-performance computing and “all markets that are challenged by data growth,” including financial services, life sciences, exploratory data analysis and chip design, media and entertainment, and manufacturing and internet of things.

Quobyte’s subscription pricing model, based on usable capacity, will remain unchanged with the 2.0 product release.

Data Dynamics StorageX 8.0 adds analysis, S3 support

The latest version of Data Dynamics’ StorageX file management software adds a single-view analysis portal, support for Amazon S3 API-compliant object storage and expanded application programming interfaces for DevOps environments.

With the StorageX 8.0 release, the Teaneck, N.J., software vendor continues to extend the product’s capabilities beyond the original data migration focus. Data Dynamics StorageX enables users to set policies and move files from one storage system to another without a gateway, file virtualization, global namespace, stubbing, sharding or agents.

Newly added support for Amazon’s Simple Storage Service (S3) API — which has become the de facto standard for object storage — lets customers move files from an NFS or SMB layer to cloud-based object storage, such as Amazon cloud storage, NetApp StorageGrid, IBM Cloud Object Storage and other S3-compliant object stores.

Data Dynamics StorageX stores file natively in the S3 object format, enabling users to attach metadata tags to ease search and management. The native S3 object format also lets applications running in Amazon directly use and work on data that StorageX 8.0 has migrated. The updated StorageX 8.0 includes a new portal to enable users to retrieve S3-archived data in an SMB file format.

File analysis capabilities

StorageX 8.0’s new analysis capabilities rely on the product’s universal data engine (UDE) to collect information on a file’s owner, size, type, access, modification and other metrics to help users make more informed decisions on data management and migration. The UDE can run on the same server as StorageX or on virtual machines (VMs) in local data center servers with management rights to access the storage, according to Data Dynamics CEO Piyush Mehta.

If you don’t know what you have, how are you making [a] business decision on what to do with the data?
Piyush MehtaCEO, Data Dynamics

“What we’ve found through our experience over the last five years is that customers just don’t know what they have,” Mehta said. “If you don’t know what you have, how are you making a business decision on what to do with the data?”

After analyzing the data, StorageX customers can use the product’s VM-based central console to set policies to enable the UDE to manage and move the data out of band. Mehta said the product is most useful in storage environments in excess of 100 TB, and the majority of the company’s 80 customers have multi-petabyte deployments across multiple data centers and geographic regions.

Mehta said all functionality provided through the Data Dynamics StorageX web-based user interface is accessible via API for management from a DevOps standpoint. An application developer can write to the StorageX APIs and gain access to the file management features and automate them based on application needs.

“You can actually integrate an application to manage storage using us as a middleware to then downstream manage the data within those storage tiers,” Mehta said.

Data Dynamics acquired the StorageX technology from Brocade Communications Systems in 2012. The company’s 80 customers span industries such as financial, pharmaceutical, telecommunications, media, oil and gas, and consumer retail, according to Mehta. He said many use StorageX with storage from NetApp, Dell EMC, IBM, Amazon S3 and traditional file servers with direct-attached storage.

Data Dynamics StorageX
StorageX analysis portal displays file characteristics, such as size, activity and age.

Data Dynamics competitors

Mehta said competitors in Data Dynamics’ management space include Avere, Panzura and Primary Data. But he claimed they use gateways, stubbing, sharding or other mechanisms that tie customers to their products to gain access to data.

“If you go with a Primary Data or ioFabric, once you allow something else to start moving your data, generally you need to keep that other [product] running so you can get to your data,” said George Crump, founder and president of Storage Switzerland LLC.

Crump said Data Dynamics StorageX doesn’t do “all the things that would set up a transparent recall,” as some of the other products do. He said StorageX moves data and then essentially gets out of the way.

Scott Sinclair, a senior analyst at Enterprise Strategy Group Inc. in Milford, Mass., said the analysis portal and native S3 object support represent another key shift for Data Dynamics StorageX, as the product expands beyond its file migration roots.

“They’re assisting beyond just the move to give you a better understanding of what type of content you have. This functionality is incredibly important when you’re looking at a hybrid cloud environment,” Sinclair said. “With the cloud, you really want to know what you’re moving because of security, compliance, performance requirements and anything else [to get] a better idea of what stays on premises versus off.”

Pricing for Data Dynamics StorageX 8.0’s new and existing capabilities is as follows:

  • $40 per terabyte for analytics;
  • $72 per terabyte for replication;
  • $100 per terabyte for file archive — SMB or NFS conversion to S3;
  • $200 per terabyte for file retrieval — S3 conversion to SMB or NFS;
  • $150 per terabyte for application modernization and transform file to S3; and
  • $500 per terabyte for file migration, for one-time use, security transformation and file-system restructuring.