©XSIBackup-DC Man Page

©XSIBackup-DC short reference manual

Always refer to the latest ©XSIBackup Datacenter Manual. It will usually be more up to date than this man page.

Read the "Ten Commandments Of ©ESXi backup" to have an overview of the most critical things to keep in mind at all time.

©XSIBackup-Datacenter is a software solution developed in plain C, having the GLibC library as its main dependency. It follows the same basic design principles of previous XSIBackup releases, namely: an efficient backup and replication tool that can be used directly in any ESXi or Linux host. As per the time of writing these words ©XSIBackup-Datacenter can replicate & backup Virtual Machines hosted in ESXi servers just like ©XSIBackup-Free and ©XSIBackup-Pro do and can also backup & replicate file structures in Linux servers both as replicas or as backups to deduplicated repositories.

Both operations may be performed to a locally accessible file system (any accessible disk or share: NFS, iSCSI, SMB...) or over IP. The nomenclature employed for IP transfers is:

user@[FQDN or IP]:Port:/path/in/remote/system

In the future we will extend ©XSIBackup-Datacenter functionality to operate on different virtualization platforms like XEN or KVM. ©XSIBackup-Datacenter is capable of performing two kind of operations on data:

- Replicas.
- Deduplicated backups.

Both of them are performed by using a simple algorithm that compares data present on the host being backed up against the data eventually already present on the remote volume. ©XSIBackup-Datacenter first detects wholes in virtual disks and jumps over them, it can also detect zeroed zones in real data and jump over them as well. Only real data present on disk is actually processed. The SHA-1 checksum algorithm is used to compare chunks of data and decide whether the block being processed must be sent to the target folder or it is already present on the other side. When zero awareness is combined with SHA-1 differential algorithm, maximum speed is reached, that is on subsequent data operations to the first run which obviously must process all non zero data.

©XSIBackup-Datacenter downloads data definitions stored on the remote side, so that all comparison operations of the XSIBackup algorithm are performed locally. Thus, all data stored remotely must not be modified in between backup cycles. In case of backups, which are always performed to a deduplicated repository, you can choose to compress data by employing the acclaimed LZJB algorithm used by Solaris and ZFS. This allows to compress data as well as deduplicate it. The use of data compression is recommended (just add the --compression argument to your backup job) it offers some 45% compression ratio. If you are backing up to an already compressed files ystem you may remove the --compression flag and improve effective transfer speed and free your CPU from the compression load.

Over IP Operations (SSH options)

©XSIBackup-Datacenter may operate in client/server mode. When you transfer data over IP, you must invoke the xsibackup binary on the other end. If you ommit the --remote-path argument, the client will look for the binary in the /usr/bin folder of the remote host. You may as well indicate the remote path by explicitly stating the remote installation path, just like you do with Rsync.



©XSIBackup-Datacenter can tunnel data through SSH to a remote server. The cipher algorithm that we may use for this tunnel can greatly affect transfer speed and CPU usage. The default set of ciphers in use is:



The above set should work well even between distant Open SSH versions, i.e.: 5.6 => 7.5 and the other way around. Its downside is that they are not very fast, unless your CPU counts with a special set of instructions to handle this workload. When ©XSIBackup-Datacenter detects that your local OpenSSH version is 6.5 or above, it will prepend chacha20-poly1305@openssh.com to the list of default algorithms, so that it's used instead of AES cipher family. This will greatly improve speed due to the nature of this algorithm, created by Prof. Daniel Bernstein when compared to AES if you don't count with some sort of hardware cryptographic co-processing help. You can optionally enable the L switch in --options=L (Less Secure Algorithm). It will try to use



This last set is comprised by deprecated algorithms, you may want to use them when you don't need that much security in encryption, like in a controlled LAN, or when you need compatibility with older OpenSSH versions. On addition to the above, you may pass your own preferred algorithsm to be used to cipher SSH data:

Basic Usage:

Basic usage consists in passing an action first plus one or two paths depending on the type of action being performed, then the rest of arguments.

./xsibackup [action] [source] [target] [options]

Quick examples:

ATTENTION: don't copy directly from this document into your SSH client. The chances that some character substitution happens is very high. Paste first to some notepad program where you can tweak the page code and see the result before pasting to the SSH terminal.



A list of all the arguments and options available in © XSIBackup Datacenter

Actions:

Actions are always the first argument in any ©XSIBackup-Datacenter job. They may sometimes include an option passed after an equal sign but most of the times they work alone.

1. --backup: this action initiates a backup operation. You will need to add, at a minimum, a backup source (VM or directory) and a backup target, namely a directory in your local file system or on a remote file system over SSH.
   
2. --replica: this action will perform a replication of the data under the source directory to the directory specified in the target argument. It will preserve the folder structure of the source.
   
3. --restore: this action will work the same way as --backup but the other way around. You point the source argument to some folder containing a backup, that is, some folder in a repository containing some .map files. The target will be the folder where to restore the contents of the source argument.
   
4. --check[=fast|full]: this action will check a whole repository or a folder inside a repository passed as the second argument, it may optionally accept two values: fast or full (default). The fast option will only check the size of the chunks in the repo. The full option will eventually uncompress the data and recalculate its SHA-1 checksum to be compared with the stored value. Please, note that when the repository is compressed, even if you don't choose the full option, every chunk's inner header will need to be queried to find the size of the uncompressed data.
   
5. --remote-path: this is the path in the remote system that allows XSIBackup to find its counterpart. You can omit it if XSIBackup can be found in the /usr/bin directory. If your remote target is a Linux server, you'd better place the xsibackup binary in /usr/bin, if on the contrary the remote host is an ESXi server, install to a persistent path, i.e.: /vmfs/volumes/datastore1/xsi-dir and pass this remote path in this argument.
   
6. --prune: this action will delete some folder inside a deduplicated repository and calculate which blocks belong exclusively to that backup, it will then delete those blocks only, liberating the space used by that particular backup in the deduplicated repository.
   
7. --info: just pass the root of a repository as a second argument to this action to have a quick view of the most relevant figures: blocks in the repository, size used, size of the files hosted and the achieved compression ratio.
   
8. --repair: using this action on a previously existing repository will perform the following: all constituent blocks for all files stored in the repository will be read, sorted and any duplicates will be removed, then each block will be sought in the /data folder to check it exists. This action can repair a repository which block manifest /data/.blocklog has been erased and rebuild it from the individual .map files, but it cannot recover deleted blocks or rebuild missing .map files. Unless you have lost fundamental data (data blocks or .map files), if a --repair operation is successful, you can consider your repository to be healthy.
   
9. --update-cron[=user]: (optional) copies the user crontab at var/spool/cron/[user]-crontab to the ESXi crontab at /var/spool/cron/crontabs/[user]. This option only works in ESXi hosts.
   
10. --install-cron[=user]: (optional) adds a line to the /etc/rc.local.d/local.sh file in ESXi hosts so that the [user] crontab is enabled upon restart of the ESXi host.
    
11. --uninstall-cron[=user]: (optional) removes the line from /etc/rc.local.d/local.sh belonging to the [user]. The ESXi crontab remains untouched. Modify the ©XSIBackup crontab and run --update-cron to completely remove the existing schedules or reboot the host.
    
12. --add-key: this action will grab the local RSA public key and place it in a remote server's authorized_keys file to allow passwordless communications between the local and the remote system. You need to perform this for every system that you want to use over IP, it's not needed for local data operations. You do need to use this kind of string as the target: user@IP:Port. The user will be root most of the times, i.e.:
    
    
    root@192.168.12.3:22
    
    
    The key exchange routine will look for authorized_keys files in the default locations for OpenSSH servers, namely: /root/.ssh/ authorized_keys, /home/some-user/.ssh/authorized_keys or /root/.ssh/authorized_keys.
    
    In case of remote ESXi hosts, the default location for this file will be sought, that is:
    
    /etc/ssh/keys-root/authorized_keys or /etc/ssh/keys-[some-user]/authorized_keys in case of using some user other than root. Examples:
    
    
    ./xsibackup --add-key root@192.168.1.221:22
    ./xsibackup --add-key alice@192.168.1.221:5467
    ./xsibackup --add-key bob@192.168.1.50:5467
    
    
    1/ Add key to host 192.168.1.221 on port 22
    2/ Add key to host 192.168.1.221 on port 5467
    3/ Add key to host 192.168.1.50 on port 5467 on user bob profile
    
    

Source:

This is the second argument in case of performing a --backup or --replica action and the only path required when executing a --check, --info or --prune operation. When performing copy operations (--backup or --replica) this argument must point to an existing directory containing some files. Those files will be backed up or replicated to the target directory.

In case of using ©XSIBackup-Datacenter in the ESXi Shell, it can detect VMs by name. Just parse a list of virtual machines between parenthesis preceded by the word VMs, like:

Target

Target is the third argument in the command line. It represents a directory where files will be backed up into an existing deduplicated directory, or replicated to it. If the directory does not exist it will be created and eventually, if action is --backup, a new repo initialized.

Options

1. --block-size[=1M|10M|20M(default)|50M]: (optional) this is the block size that will be used to deduplicate data when using the --backup action. In case of replicas a fixed block size of 1M will be used. You can choose between: 1, 10, 20, or 50 megabyte block sizes when performing a --backup action. 10M is the default --block-size.
   
   
2. Example:
   
   --block-size=1M Set block size to one megabyte
   --block-size=20M Set block size to twenty megabytes
   --block-size=50M Set block size to fifty megabytes
   
   
3. --quiesce: (optional) this option has no arguments, use it when backing up VMs to quiesce the guest OS previous to taking the backup snapshot. If you don't pass this option, no quiescing will take place.
   
4. --compression: (optional) sets whether the backup will compress chunks previous to storing them in the backup repository. It will achieve an additional 50% compression on the data at almost no cost in speed. It's recommended that you always use it, unless you are storing data to a compressed file system, you may avoid using it in that case.
   
5. --subfolder[=yourfolder]: (optional) this will add a folder with the specified name before the time-stamped folder, so that you can organize your backups in logical containers. It's useful when storing backups from different clients into a consolidated repository per instance.
   
6. --rotate [=ND]: (optional) this option will delete the backup folders older than N days when the number N is followed by a D letter (case insensitive), or will just keep the number of folders stated in the numeric value when you parse just an integer number.
   
7. --verbosity: (optional) accepts values between 0 and 10, more information messages will appear on STDOUT when you raise it.
   
8. --save-job[=NNN]: (optional) will save the current command line job to a job file in etc/jobs/NNN, where NNN is the three digit numeric code assigned to the job. You may then call this file from the host's crontab to schedule backups or replicas.
   
9. --timestamp [=YYYYMMDDhhmmss]: (optional) allows to set a custom timestamp in case you want to place multiple job outcomes into a single backup folder. If you don't use it, XSIBackup-Datacenter will create a timestamped folder per backup job.
   
10. --ssl-key [=path]: (optional) path to the SSL key pair. If it is not provided by means of this argument it will be sought in the program's root folder.
    
11. --exclude [=REGEXP]: (optional) regexp pattern to exclude disks from the backup. The REGEXP can be up to 512 characters long. --exclude="vmware-[0-9]{1}.log"
    (Excludes all log files like: vmware-2.log, vmware-2.log, vmware-2.log, etc...).
    
12. --ssl-key [=path]: (optional) path to the SSL key pair. If it is not provided by means of this argument it will be sought in the program's root folder.
    
13. --mail-to[=some@email.com]: this is the e-mail address where the e-mail reposrt will be sent.
    
14. --use-smtp[=N]: this is an integer number starting with 1, that makes reference to an SMTP entry in the SMTP servers configuration file at [xsi-dir]/etc/smtpsrvs.conf. In case you don't provide one, the e-mail submission routine will fall back to SMTP server number 1.
    
15. --subject[=some subject]: sets the subject text of the e-mail report. In case you don't provide one a generic subject text will be displayed.
    
16. --html-template[=NNN]: sets the template to use to embed the backup report. Classic template is included as 000.html, you may use it as a base to create your own designs. The only rule is to keep the placeholder indicator (<!-- PLACEHOLDER REPORT -->) in its own line, it will be replaced by the data generated during the backup. You may use numeric string values 000 to 999