Last updated on Friday 20th of May 2022 09:10:57 AM

©VMWare ©ESXi backup to a Linux server with granular recovery

Alert sign Please note that we provide tools to perform granular recovery like XSIGR, bundled with every package (free and paid versions) without the need to comprehend the nitty gritty details of the process. This post is here for more advanced users willing to comprehend every step and/or apply the logic to other virtualization systems.

  • Contents

  • General considerations

    With the advent of ©XSIBackup-DC we have reached maximum flexibility and feature offer for the enterprise by adding granular recovery of individual files within an unlimited set of recovery points which are in turn deduplicated and compressed and are generated in a differential manner.

    Thus, by using ©XSIBackup-DC you can generate a virtually unlimited set of backups in a volume with barely twice the size of the amount of data to backup and you can restore individual files from any of the restore points really easily.

    Installing ©XSIBackup-DC to your ©ESXi box is a trivial task taking some seconds, just copy the program's folder and assign permissions, that's all.

    Once you have installed the software creating your first backup is also a kiddies game.

    ./xsibackup --backup "VMs(RUNNING)" /vmfs/volumes/BACKUP

    That's all. The rest of parameters will be assigned by default: 1MB block size plus compression, and that's tweaked to work the best way in most systems, so you can just leave it like that.

    Now the thing is what that BACKUP datastore highlighted in upper case should be to achieve the best results.

    If you just use a regular HD or SSD attached to your local controller as a backup target, you are going to hit some limitations: first of all you can only format that drive as a VMFS volume, which has a limited number of i-nodes, around 130.000 for VMFS-5 and an undetermined number for VMFS-6 (VMWare just doesn't publish information on this figure), so you won't be able to store much data, unless you use a big block size, still not the best approach.

    Apart from that, storing backups to a local hard drive would not be a good backup methodology from a basic conceptual point of view, as any physical problem affecting the server would in turn affect your backups too. The most basic approach to backing data up tells you: the furthest away the better, that's why backups in the cloud have become so popular. You may also keep a reduced set of local backups, namely in a fast LAN, to speed recovery up in case of disaster, nonetheless any backup should be out of your server.

    ©XSIBackup-DC allows to backup your data to any local datastore as well as to any IP network location which is reachable by SSH protocol, making it very convenient to perform backups in the cloud, as it's extremely efficient doing differential backups.

    Ext4 or XFS file system is better Best local datastore type

    The best kind of local datastore to keep your backups is an NFS volume, as you can enjoy the benefit of choosing the underlying file system in the NFS volume. Most general purpose file systems out there are able to store millions of files, nevertheless we recommend that you use ext4 or XFS, as we have found them to be extremely reliable as well as really fast.

    Remote file systems

    In regards to remote file systems the same applies. You can use ©XSIBackup-DC in any Linux OS that can execute the binary, which has a wide compatibility range. You should be able to execute it in Linux versions as old as RHEL 5 x64. In regards to the file systems you use to format your backup volumes, as said, use ext4 or XFS if you can choose, any other decent FS will do it though.

    You may use some FUSE based filesystem as GlusterFS or ZFS, but please be aware that activating features such as deduplication will be useless and will turn down performance so much that you will end up with a clogged system.

    Installing the server software

    This procedure applies to Linux servers mounted over NFS or over IP. The only difference is that if you mount some Linux server over NFS, you won't need ©XSIBackup-DC installed on it, whereas if you backup your VMs to an IP backend server you will need it installed in the remote end to act as an IP server.

    We will start with a clean minimal install of CentOS 7 in its most recent version 7.7 (1908) and will add the additional components required to mount backups as .vmdk disks and access their data to restore individual files. The IP configuration is the default one, there's no need to add a static IP as most Sysadmins will use the DHCP server to assign static IPs, so we'll leave the server with a default automatic ipv4 and ipv6 configuration.

    Before anything else we update the system and install the EPEL repository for CentOS.

    # yum -y update; yum -y install epel-release

    We will turn off Selinux and ipv4 and ipv6 firewalls to make sure that they don't interfere with our set up. Once we have everything working as we want we can protect the backup server by enabling Selinux and allowing only the IP traffic we want.

    sed -i '/^SELINUX=enforcing/c\SELINUX=disabled' /etc/selinux/config;
    systemctl stop firewalld.service;
    systemctl disable firewalld.service;

    First thing to do is install ©XSIBackup-DC binary xsibackup and the ©XSIBackup File System binary xsifs. Doing so is so easy, you just have to copy both binaries to the /bin folder in the server. To do so you can use scp from a Linux command line or WinSCP from any Windows desktop.

    Once you have placed both binaries in the /bin folder of the server, just assign appropiate permissions on the so that you can execute them with any user.

    chmod 0755 /bin/xsibackup /bin/xsifs

    And execute both of them without any argument just to check that you can see the help on screen. You will see an error like the one below when you execute xsifs, we are lacking Fuse3 client, which is a dependency for xsifs.

    xsifs: error while loading shared libraries: cannot open shared object file: No such file or directory

    To install Fuse3 just issue the following command. The package is available in the EPEL repository, thus it should be installed straight.

    yum -y install fuse3

    Using XSIFS File System

    Now you will be able to execute xsifs without arguments and get the help on screen.

    [root@xsinas2 ~]# xsifs
    ©XSIBackup File System
    ©33HOPS, Sistemas de Informacion y Redes, S.L.
    ©Daniel J. Garcia Fidalgo |
    Usage: xsifs --backuppath=/path/to/backup

    Use some backup path in an ©XSIBackup-DC repository as the first argument
    The contents will be mounted in the argument as a file system

    Not only that, since this moment we are already able to mount any ©XSIBackup-DC repository in any Linux folder at this server. So, from your ©ESXi host just exchange keys by using the --add-key argument from the xsibackup client with the Linux server we just prepared and backup some VMs to this host. In our case we are going to backup some test VMs to the /home folder into DC repositories with different block sizes.

    In our case we chose the mount point /mnt/xsifs to mount the ©XSIBackup-DC repositories. Let's mount our first repository stored in /home/dc-repo-WC-1M.

    xsifs --backuppath=/home/dc-repo-WC-1M /mnt/xsifs

    Now we are able to access the files contained in the repository as if they were real files. In fact by virtue of the xsifs file system they are indeed real files. Until here nothing spectacular, you can achieve the same result by restoring some backup, the real advantage is that by using this method you don't need to restore a backup to access its contents, you can do so instantly.

    So you can keep many terabytes of data in a repo which is some hundreds of gigabytes in size per instance, but still be able to access individual files in real time at any of the backups.

    [root@xsinas2 ~]# ls -la /mnt/xsifs/20200219175503/CentOS-7.2/
    total 0
    drwxr-xr-x. 2 root root 0 Feb 19 18:57 .
    drwxr-xr-x. 2 root root 0 Feb 19 18:57 ..
    -rw-r--r--. 1 root root 45056 Feb 19 18:57 CentOS-7.2-000001-delta.vmdk
    -rw-r--r--. 1 root root 323 Feb 19 18:57 CentOS-7.2-000001.vmdk
    -rw-r--r--. 1 root root 45056 Feb 19 18:57 CentOS-7.2-000002-delta.vmdk
    -rw-r--r--. 1 root root 330 Feb 19 18:57 CentOS-7.2-000002.vmdk
    -rw-r--r--. 1 root root 45056 Feb 19 18:57 CentOS-7.2-000003-delta.vmdk
    -rw-r--r--. 1 root root 330 Feb 19 18:57 CentOS-7.2-000003.vmdk
    -rw-r--r--. 1 root root 13 Feb 19 18:57 CentOS-7.2-aux.xml
    -rw-r--r--. 1 root root 21474836480 Feb 19 18:57 CentOS-7.2-flat.vmdk
    -rw-r--r--. 1 root root 8684 Feb 19 18:57 CentOS-7.2.nvram
    -rw-r--r--. 1 root root 19895 Feb 19 18:57 CentOS-7.2-Snapshot10.vmsn
    -rw-r--r--. 1 root root 19895 Feb 19 18:57 CentOS-7.2-Snapshot11.vmsn
    -rw-r--r--. 1 root root 19888 Feb 19 18:57 CentOS-7.2-Snapshot9.vmsn
    -rw-r--r--. 1 root root 551 Feb 19 18:57 CentOS-7.2.vmdk
    -rw-r--r--. 1 root root 1073 Feb 19 18:57 CentOS-7.2.vmsd
    -rw-r--r--. 1 root root 1073 Feb 19 18:57 CentOS-7.2.vmsd.tmp
    -rw-r--r--. 1 root root 2866 Feb 19 18:57 CentOS-7.2.vmx
    -rw-r--r--. 1 root root 2866 Feb 19 18:57 CentOS-7.2.vmx.tmp
    -rw-r--r--. 1 root root 341417 Feb 19 18:57 vmware-1.log
    -rw-r--r--. 1 root root 244539 Feb 19 18:57 vmware-2.log
    -rw-r--r--. 1 root root 198192 Feb 19 18:57 vmware.log

    As you can in turn keep an unlimited number of restore points in your repository, this means that you have instant access to any version of your files inside your VMs in any of the restore points or backups contained in the repository, isn't that great?.

    We still need to read the .vmdk disk to access its contents, we'll leave that for the next section though.

    Next: Access .vmdk disks in Linux

    Daniel J. García Fidalgo