in the post
https://33hops.com/xsibackup-faster-dat … ckups.html
you explain in detail how the backup hardware should be designed to achieve a good performance.
Many thanks for that!
What I am missing is some real life examples.
Could you give some examples for some real computer and network hardware configuration used and
the speeds that where achieved with it?
This could give some reference to compare ones own hardware and data rates to.
We constantly mention our reference set of hardware in our lab. It's nothing but commodity hardware:
- i3/i5/i7 CPUs, most of them are fairly old > 5 years.
- NAS: Synology and Linux (CentOS & Ubuntu).
- Switches: Microtik
- NICs: Intel, from multiple port server NICs to cheap Intel desktop ones.
- HDs & SSDs: we have tried a wide variety of HDs and SSDs. The ones that we use the most are Seagate from 1 TB to 4TB, we use Ironwolf with a bigger cahe for the Synology and Linux NAS appliances.
With the above mentioned hardware you should expect speeds ranging from 70MB/s when backing up over IP on a Gigabit NIC to over 100MB/s on local disk to disk transfers. Those figures are for non zero data, i.e.: some disk which is 100% full.
We use cheap controllers, you should expect an increase in speed when you use some array controller mounting some cache. In fact enabling cache in this type of controllers is a must.
Now the main thing to take on account is that if you choose just one thing wrongly, you will set a bottleneck and limit the whole set's speed. Per instance, if you use some cheap gigabit switch from one of those brands that we sometimes use for non critical things and work like a charm in those cases, you will get extremely poor results <20MB/s
If you coose some cheap NIC, I won't mention brands, you're done.
The key is the network hardware, you can't save on that. Still, you have really good stuff at bargain prices, we are pointing you to what works for us, still every master has his small book of secrets.
Another very important thing is the use of SSD as cache. (c)ESXi allows to add SSDs as cache for HDs, do it.
We'll dedicate some time to write some posts on how to build the "perfect" NAS device to store backups for under 100.00EUR/USD
It will obviously imply using Linux HDs + SSDs in a cache configuration optimized for writes.
I use NFS storage forEsxi and I have around 20MB/s for backup with xsibackup-dc. My network have nics with 10Gb/s and hdd around 200MB/s. If I use cp I have on same NFS 200MB/s.
Can you give me a command optimezed for speed?
I have a DC version now.
Last edited by daniel_ham (2021-11-23 09:27:52)