I am using DC 18.104.22.168 to backup VMs of a customer from a SSD datastore (SAS) to a "local" NFS datastore exported by a local Synology NAS (RS1221). Esxi is running on an HPE DL360g10 machine. Using 1G ethernet to reach Synology that connects via fiber 10G to a Cisco 550XG switch. Latency is 0.4-1ms.
If I go for a --replica I see results that I would expect: 75-85 MB/s.
If I go for a --backup, even the first shot, on a new repository (empty) I only get 18-24 MB/s. That's a bit disappointing.
I would like to know where could be "the catch".
I found a post in this forum https://33hops.com/forum/viewtopic.php?id=753 where you state that latency is the key factor here.. which I do understand. But I cannot think why there's such a big difference between replica and backup. You state in another post that sha-1 calculation is cpu cheap so not that the guilty. So what? Maybe compression which is enabled by default?
In the same post you state that XSI works differently when going SSH to a NAS because it loads all blocklog db entries instead of checking them one by one. Do you think going SSH instead of NFS will improve in my scenario? [I am a bit reluctantly because as you stated Synology is closing down NAS so early a DSM upgrade could be show stopper] And if so, why not add also an option to enable that in "local datastore mode" ?
PS: sorry I see that maybe this is a topic for the speed section...
Last edited by michib (2021-08-28 14:47:36)
Thank you for your feedback.
As per current design, a local datastore, and that can be some NFS/iSCSI volume, is considered a local FS, with all its consequences, being the most important one that latency in general (disk+network) iwill be considered to be that of some local device.
Depending on your network hardware and your real scenario, that network latency added to I/O operations can sum up and reduce your effective I/O speed as you describe, as the backup behaviour is synchronous for local backups.
This does not affect replicas, as they are performed in a totally different way: replicas write some file's data to the same remote file, there aren't any seek times or file creation times associated to I/O as there's only a block to write to, namely: the VHD file.
The way to improve speed if you need more is to perform the --backup action over IP or to reduce the datastore access latency by using a locally attached device (some local hard disk).
The best target for some over IP backup is a Linux FS. Devices like Synology or QNap are great for novices, or if you need some easy to configure NAS, still, they won't allow you to configure them as a Linux box would do. They are propietary environments and they seem to have the will to become even more propietary in the future.
We offer an easy to deploy virtual appliance that you can use to configure some backup VM over virtual disks:
Or you can use any distro of your choice. CentOS or Debian are the best choices for us.
In regards to adding local storage attached to a local controller: as you probably already know VMFS-6 is not a good FS to store deduplicated backups, primarily because it's slow, it wasn't designed for speed. Thus you need some XFS or ext4 FS to store the backups.
A good way to have a local disk with some XFS or ext4 file system is to create some locally attached RDM disk with vmkfstools
vmkfstools -z /vmfs/devices/disks/t10.ATA_____ST500DM0000022DD142__________________________________ _W2APL5M4 /vmfs/volumes/datastore1/MY-LINUX-VM/RDM01.vmdk
Add it to the Linux virtual machine.
Then format it as XFS or ext4 from the Linux VM, this will allow you to access some locally attached disk that you can mount as NFS or access via IP from within the local IP space of the (c)ESXi host, with almost no latency and 10GB virtual NICs.
We will write a post on how to prepare such device in some days.