©XSIBackup user question:
On a Gigabit network, what is the expected throughput?
Please note that this post is relative to old deprecated software ©XSIBackup-Classic. Some facts herein contained may still be applicable to more recent versions though.
For new instalations please use new ©XSIBackup which is far more advanced than ©XSIBackup-Classic.
There is some common extended belief about some sort of limited throughput from the ©ESXi shell, as the NIC throughput is far less than expected, in any case there does not seem to exist any confirmation from part of ©VMWare and the matter itself looks like being related to encryption CPU usage. The ©ESXi Shell uses only one of the available cores, thus encription can be a serious bottleneck when transferring big files over SSH, as xsibackup-rsync does. The solution: more powerfull individual cores, which is difficult to get in a short term and low budget (see update at the end of the page).
We use pretty old hardware for our tests and development (this pushes us to try to optimize everything a bit more) and get transfer speeds between ESXi hosts (rsync shell to shell) of around 30 mb/s (we have updated this last figure in Jan 2017) using two i3 with regular Seagate hard disks and a 120 gb. SSD cache disk. This is far less than expected from an Intel gigabit lan which is our reference NIC (we use desktop and server NICs with various chipsets: 82574L, 82576). As the CPU keeps relatively low and the disks throughput is far beyond that empiric limit, there seems to be a clear bottleneck on the network speed achieved from within the shell. On rsync host to host syncronizations the first load of data will be fairly slow, but subsequent updates will be a lot faster, about twice as much.
If you are running this in a
LAN you can always make the first copy by other means.
Take on account that the preferred backup method is to a datastore, this will ensure a really fast copy. Rsync transfers
are an alternative for those standalone servers sitting in a hosting environment where you cannot link to a datastore
properly or for people wishing to maintain a fresh copy for "a kind of" failover strategy.
In any case my first attempt, even in a locked-down environment like an ESXi dedicated server, would be to try to link to an
NFS server and do a regular copy as this will be a lot faster. If you connect to a remote NFS server over a WAN, you will need
to activate asyncronous NFS for the mount to be usable. If I had limited bandwith and/or some unpredictable network
instabilities I would then go for a Rsync strategy.
We have added many new options and file copy programs to ©XSIBackup-Pro Classic in 2016 and 2017, from v 6.0 to v. 8.0. Now you can use Borg Backup as a data backend and obtain effective speeds over 50 mb/s with modest hardware and take advantage of combined deduplication and compression. If you use OneDiff you'll be getting effective average speeds over 700 mb/s, calculated over the total size of the file, only the differential data is really transferred though. In 2017 we'll be adding our own propietary file copy and replication mechanism, which will offer native block level deduplication on VMFS and other FSs attached via a DS, and block level de-duplication differential copy over IP.
Daniel J. García Fidalgo
33HOPS
