About Deep Storage

Deep Storage, LLC. is committed to uncovering the underlying truths about storage, networking, and other data center technologies, aiming to assist information technology professionals in providing top-tier services to their users while maintaining a reasonable work-life balance. Deep Storage Reports are the culmination of our hands-on testing and over 30 years of experience in implementing technology in real-world scenarios.

Our approach to testing is grounded in real-world usage, ensuring that systems are configured in a manner reflective of how most customers would utilize them. This strategy avoids the creation of artificially optimized configurations designed solely to maximize benchmark performance, a practice often referred to as "Lab Queen" configurations.

While this report was sponsored by our client, it's important to note that Deep Storage always maintains full editorial control over our publications. This ensures that our reports remain unbiased and objective, providing valuable insights to our audience.

Introduction

When the IT market adopts a new platform, the operational teams tasked with its maintenance must navigate the challenge of implementing a backup strategy. While they could integrate the new platform into their existing enterprise backup solution, these solutions often lag in leveraging platform-specific APIs and optimization for the new technology. This lack of optimization can result in sluggish backups and a notable impact on system performance.

During the nascent stages of a technological revolution, the market may not be sizable enough for established enterprise players to address it adequately. This creates an opportunity for more agile startups to step in. For instance, in the RISC/UNIX/open systems era of the 1990s, we saw the emergence of NetBackup, NetWorker, and TSM—today's dominant trio in the enterprise backup landscape. Similarly, ArcServ and Backup Exec initially focused on backing up NetWare servers but experienced significant growth during the Windows Era.

Most recently, the move to vSphere spawned vRanger, PHD Virtual, and of course Veeam.

The cycle repeats itself as Nutanix advocates for its in-house AHV hypervisor. In the early stages of AHV adoption, users found themselves reverting to outdated backup methods, resorting to installing agents to back up VMs. Recognizing this gap, the team at Comtrade Software seized the opportunity and developed HYCU, the pioneering backup application specifically designed to safeguard AHV systems, seamlessly integrated with AHV's snapshots.

While other backup vendors have begun to offer basic support for AHV in recent months, HYCU's unwavering commitment to this platform ensures it provides broader support for Nutanix features at a faster pace than its competitors.

In this Technology Validation Brief, we delve into HYCU's support for Nutanix Files, the file services feature of the Nutanix platform. As of February 2019, when this brief was authored, HYCU stands alone as the sole backup application supporting Nutanix's Changed File Tracking API, further solidifying its position as the go-to solution for Nutanix users seeking comprehensive backup capabilities.

The Bottom Line

HYCU Data Protection for Nutanix (HYCU) is the first platform-specific backup solution for Nutanix’s fast-growing HCI infrastructure.

While enterprise backup applications provide some support for Nutanix’s AHV hypervisor and Prism management server, platform-specific solutions such as HYCU have historically been easier to use, and offer tighter integration with their selected platforms.

With HYCU this tighter integration results in:

• HYCU-managed Nutanix snapshots as restore points
• Automatic protection of newly created VMs with default policy
• Support for Changed File Tracking
• Eliminates the overhead of file system scans
• Significantly speeding incremental backups

Testing of HYCU to protect a 3.8 million file, 587GiB revealed:

• Policy definitions aligned to business-like RPO/RTO
• Policy compliance includes estimated restore time
• Incremental backup file enumeration <10 sec
• No noticeable performance impact

Organizations planning investment in Nutanix infrastructure should take a good look at HYCU to take full advantage of that investment.

File Services: HCI’s Missing Link

Hyperconverged Infrastructure (HCI) solutions leverage software to transform the storage media within hypervisor hosts into a resilient storage pool accessible by the virtual machines (VMs) running on those hosts. However, many HCI solutions lack native file services functionality. While VSAN clusters offer ample space for VMs, organizations often require file shares, which necessitates another solution—an indispensable need across various applications, including Virtual Desktop Infrastructure (VDI) deployments.

Even VDI, which is considered a quintessential HCI application, relies on reliable, high-performance file services to manage users' persona information and data files effectively. Some organizations opt for running virtual Windows file servers or file server clusters. While Windows servers offer robust functionality, they often require additional management and patching efforts, complicating cluster operations. Alternatively, some organizations rely on enterprise NAS filers, missing out on HCI's promise of eliminating costly, dedicated storage hardware.

Enter Nutanix Files

Nutanix Files, formerly known as Acropolis File Services, provides SMB (2.0,2.1 and 3.0) and NFS (v4) services using three or more virtual machines to create a scaleout file server. Nutanix Files does a better job supporting Windows clients than most Linux-based NAS systems. Nutanix Files uses proper NTFS ACLs and even supports the Windows file explorer’s previous versions feature to empower users to restore from snapshots themselves.

File servers and shares are created and managed via the Prism UI, which makes Nutanix Files significantly easier to manage than a Windows file server cluster running as VMs. To create a new file server you just enter the VM IP addresses and size into a wizard and wait for Prism to do all the hard work.

By providing SMB/NFS access to the Nutanix cluster’s distributed data store, Nutanix Files fills an obvious gap in the HCI’s feature set.

Introducing HYCU

As is only appropriate for a backup application that protects a hypervisor’s resources, HYCU is distributed as a virtual machine image. This saves administrators from the drudgery of spawning a VM, applying patches and pre-requisites like runtimes, and installing the backup application while ensuring the VM includes all the libraries and packages HYCU depends on.

Once the image file is deployed and assigned an IP address, HYCU’s HTTP interface makes it simple to complete the configuration, including connecting the backup controller to Prism or vCenter, and operating HYCU.

To start protecting data, just set up a backup target on an SMB or NFS share such as another Nutanix Files cluster, Azure, an S3-compatible object store, or an iSCSI LUN. (At present you can’t use tape or virtual tape libraries, which is probably a good thing.) Then you just have to assign backup policies to the VMs, file shares, and applications that HYCU discovered in Prism or vCenter and direct them to a target.

Modern Backup: Policies Not Jobs

Traditional backup applications center around backup jobs. Administrators have to decide how often to perform full and incremental backups and then assign a schedule to each type. Pretty soon the backup application’s console becomes a schedule of hundreds of jobs with any dependencies between them buried in notes and institutional memory.

More modern backup solutions like HYCU use policies. Administrators specify the frequency, retention, and placement of backup copies to the resources to be protected. Defining a policy only requires specifying the backup frequency, which correlates to RPO, a recovery time target, or RPO and a retention period.

HYCU tests the backup and restore performance as it performs each copy, and reports a resource as being out of compliance with its policy not only if a backup job fails but also if HYCU projects that a restore of this resource will exceed the policy RPO.

Rather than scheduling full and incremental backups on a static schedule, HYCU policies specify a change threshold. HYCU runs a full backup when changes exceed 25% in the default profile. That alone should reduce backup traffic significantly on the large amounts of cold data in most corporate data centers.

Some of the more advanced policy options reveal the close integration between HYCU and the AHV infrastructure. The Fast Restore option, for example, manages how long Nutanix snapshots are retained, while the Backup from replica option creates backups from replicated copies of remote VMs.

Snapshots and Changed File Tracking
- The Modern Way to Backup Files

Traditionally, backup applications have initiated incremental backups by scanning the file system's metadata, examining attributes such as the last modified date or archive bit to compile a list of changed files requiring backup. However, this file system scan process is resource-intensive and can significantly impact real-time user access to files, often resulting in prolonged backup durations.

In scenarios with large file systems, the initial file system scan at the start of each backup job can sometimes exceed the time taken to copy the relatively small number of changed files to the backup repository. This delay can lead to instances where users struggle to meet their daily incremental backup Service Level Agreements (SLAs), as the file system scan alone may consume more than 24 hours.

To circumvent this challenge, Nutanix has introduced changed file tracking APIs into Nutanix Files. Instead of relying on traditional file system scans and file copying methods via SMB or NDMP, HYCU leverages these APIs to execute a more efficient backup process.

When HYCU invokes the Nutanix backup APIs, Nutanix Files generates a snapshot of the file system and provides HYCU with access not only to the snapshot data but also a comprehensive list of files that have changed since the last invocation of the backup API by any application. This innovative approach ensures quicker and more streamlined backups, enhancing overall backup performance and minimizing disruptions to user access.

Conventional file scans can have a significant impact on NAS performance over many hours for each incremental backup.

The snapshot provides a consistent, point-in-time version of the file system, which prevents two other problems with old-fashioned backups--open and inconsistent files. Standard file access protocols like SMB don’t allow an application to access files a user has opened for writes. But some files, like Outlook PSTs are kept open, and therefore locked, as long as their applications are open. If a Sr. Executive VP never closes Outlook, his very important PSTs will never get backed up.

More insidious are inconsistent backups. A conventional backup may take minutes or hours from the time the first file is backed up to the completion of a job. If an application makes changes to two or more files while the backup is running, the backup system may backup one file before the group is modified and other files after the change. This makes the group inconsistent. Since Nutanix Files creates a snapshot of the whole file system at an instant in time, all the files are both available and consistent.

Once HYCU has access to the changed file list and the snapshot, it uses those Nutanix backup APIs to access multiple files in parallel so the data can be copied to safety as fast as the backup target can accept it.

Today, HYCU is the only backup application to fully leverage these Nutanix APIs to eliminate the overhead of a file system walk, produce complete and consistent backups, and move data in parallel. Compared to more conventional solutions, HYCU backups should run faster and reduce the impact on other application.

How we tested - how HYCU protects Nutanix Files

HYCU’s integration into the Nutanix ecosystem minimizes the overhead of protecting AHV VMs and makes it easy to maintain approximately two days of snapshot history on the Nutanix cluster to provide quick restores of full VMs, or individual files. But VM backup is just the first step in supporting the full Nutanix ecosystem.

We wanted to explore how HYCU protects Nutanix Files. Our experience tells us that while it’s easy to provide basic protection of an SMB share, getting file system backup right requires greater integration between the file system and backup application.

First, we created a dataset to protect. The folks at HYCU gave us remote access to a 3 node Nutanix AHV cluster in their Boston headquarters, complete with a Nutanix Files server and HYCU instance. They also gave us access to a backup target within the HYCU environment.

Because actual backup performance depends on so many factors, and we were working on shared infrastructure, we believe the 15MB/s backup rate we saw in testing could be improved with a little tuning to HYCU and/or the generic backup target.

We created a share on the AFS (Acropolis File Server), mounted it from a Windows Server 2012 virtual machine, and ran a script to create a test file system.

The resulting file system contained:

• Files               3,831,596
• Folders          91,438
• Total size      237.3GiB or is it 586.23GiB

The initial dataset size, comprising a substantial number of files and significant total size, renders both file system traversal for changed files and frequent full backups impractical. To assess this, we devised a backup policy aimed at backing up our file system every two hours. Subsequently, we initiated a script simulating user activity, creating new files, appending to existing ones, deleting files, and mimicking a day's worth of work on a two-hour schedule.

After running this emulation for a couple of days, approximating a month's worth of user activity, we evaluated the outcomes. Our script was designed to generate approximately 5% new data with each iteration. Upon examination, we observed that every fifth or sixth backup entailed a full copy, aligning with the 25% threshold specified in our policy.

Additionally, we were intrigued by the speed at which HYCU cataloged the file share—a common pain point in traditional backup solutions. Remarkably, HYCU consistently completed cataloging the share in less than a minute from job initiation to actual data movement to the generic backup target, with the Nutanix system's snapshot creation consuming most of this time.

Notably, regardless of whether the dataset comprised 100 files or 100,000, HYCU consistently required no more than 10 seconds to catalog the share and commence data transfer once the snapshot was ready. This efficiency underscores HYCU's effectiveness in handling backups, even with substantial datasets and frequent backups.

We Backup to Restore

More specifically, we backup file shares to be able to restore the files our users, and their applications, have overwritten, renamed, and otherwise deleted. Restoring files with HYCU is simple.

First, choose the share to display the various restore points available.

Select a restore point and click the ‘Browse and restore’ button to open a tree view of the share at the time of the selected backup.

Select the file(s) you need, then choose if you want the restored files redirected or renamed. Voila! That important contract your CEO “misplaced” is right where it should be.

The ‘Export share’ button, which we didn’t test, will restore the whole share to another file server/NAS.

Conclusions

HYCU serves as a prime example of the benefits offered by a modern, policy-centric backup application seamlessly integrated with the platform it safeguards. Administrators can adopt a business-centric approach, defining policies based on Recovery Point Objectives (RPOs) and Recovery Time Objectives (RTOs) relevant to business needs, while monitoring compliance with these policies.

The close integration with Nutanix enables HYCU to manage Nutanix snapshots as restore points, automatically assign newly created VMs to default policies, and utilize Nutanix's Changed File Tracking (CFT) API for backing up Nutanix Files shares with minimal impact.

During our testing of HYCU's protection for Nutanix Files, we found that it could identify changed files for incremental backups within ten seconds, even when altering just 100 files in a file system boasting over 3.8 million files. This is a remarkable achievement compared to conventional backup solutions, which may take hours and exert significant pressure on the file system while scanning metadata.

Organizations investing in Nutanix infrastructure would be wise to consider HYCU. Historically, platform-specific solutions like HYCU have been quicker to support backup-enabling APIs such as CFT, typically six to eighteen months ahead of enterprise backup providers. This underscores HYCU's agility and alignment with Nutanix's evolving ecosystem, making it a compelling choice for Nutanix users seeking robust backup solutions.