Last updated on Monday 28th of February 2022 08:52:48 PM

©XSIBackup-Pro 8.0.0, Support for borg backup as an IP backend

Alert!  This is an old post. Since it was written we have developed our own deduplication engine which is specifically aimed at backing up virtual disks: ©XSITools. It is lighter than using Borg for this purpose and you can accumulate many TB without suffering from an increased slow down as you add data.

The engine has also been ported to C languaje in our latest project ©XSITools Datacenter and its power has been stretched to take the hardware to its full potential.


As commented, Borg Backup is a fork of Attic Backup which seems to be a more active project for the time being. ©XSIBackup-Pro offers support for copying VM backups to any IP server equipped with a Borg Backup install. Installing Borg Backup is a trivial task most of the times. Just copy the corresponding binary i386 or x64 to your path at /bin or /usr/bin and that's it. Our reference OS is CentOS 6.8 with an ext4 FS. You are free to use the OS of your choice, but do pay attention to Borg Backup's requirements first, or you might end up with an unstable system. You can store many terabytes of real data in usually less than one tenth of the nominal amount of data and still browse and restore contents nimbly.

Making a backup

To make a backup to a Borg backend, you just need Borg Backup on the other side and full root access through the SSH port (22 is default). ©XSIBackup-Pro pipes data through an SSH tunnel, thus, all transmitted content is encrypted by using the ©XSIBackup-Pro RSA key. Data is "tarred" and sent over a pipe, this means the packed size will be slightly bigger than the sum of the files, due to tar headers and file separators. As in every other "over IP" scenario with any version of XSIBACKUP(FREE|PRO), you must first link to the other server by using the --link-srv=[IP of remote system] argument. This will ensure that transparent encrypted communication is possible from the client to the server side of the pipe.

Examples:

# ./xsibackup --backup-prog=borg:z --backup-point=10.0.0.1:22:/backups/my-borg-repo --backup-type=custom \
--backup-vms="MyVM1,MyVM2!MyVM2_3.vmdk;MyVM2_4.vmdk" --mail-from=email.sender@yourdomain.com \
--mail-to=email.recipient@anotherdomain.com --smtp-srv=smtp.33hops.com:25 --smtp-port=25 \
--smtp-usr=username --smtp-pwd=password


Where 10.0.0.1:22 is the address of a fully accessible Borg server. The [z] after the borg program name at --backup-prog=borg:z is recommended as it will not affect efficiency but will improve storage to disk usage ratio.

There's currently no possibility to run a true Borg Backup client on the ESXi family of OSs, thus all data is sent to the other end of the pipe. This won't be a problem in a LAN, but makes backups via Borg almost impossible to be performed over a regular WAN, unless the VM sizes are restrained. To backup data over a WAN use  nbsp;OneDiff to make sure you only transfer changed blocks, and once you have the consolidated and mirrored VM on the other side, then use a Borg Backup command to store a copy of the current VM. This way you can take advantage of a differential backup and an archive of deduplicated data that will allow you to move back and forth in time to get to the desired state of a given VM in the event of a disaster recovery scenario.

Storage efficiency:

In regards to this topic, nothing better than a real example of what block level deduplication plus compression can do for your storage budget. Below these lines you have an excerpt of a real Borg repository storing one week of VM backups, which are a mix of Windows 2008 Server, Windows 7, Windows XP and different Linux servers. As you can see, 1.39 Tb. are stored in just 86.94 Gb of storage.

Name: 1482770833 - esxi1.clm95.33hops.com - LCTEST1 :::
Fingerprint: cc0ada76bb297145b05d1c6d63c52d041161079c185359716c3745c95c104d39
Hostname: XSINAS
Username: root
Time (start): Mon, 2016-12-26 17:47:35
Time (end): Mon, 2016-12-26 17:50:34
Command line: borg create -v --stats --progress --compression lz4 /xsinasbak/backvmsup::1482770833 - esxi1.clm95.33hops.com - LCTEST1 ::: -
Number of files: 1

Original size Compressed size Deduplicated size
This archive: 17.18 GB 454.06 MB 324.00 MB
All archives: 1.39 TB 383.55 GB 86.94 GB

Unique chunks Total chunks
Chunk index: 55999 324697


Storage format:

As commented, ©XSIBackup-Pro stores data as a tar file. You can use the provided restore function in ©XSIBackup-Pro, or if for whatever reason you whish to manipulate the data manually, you can extract the Borg backup and untar the resulting stdin file right in the Borg server.