#1 2017-11-09 14:20:49

sistemi
Member
Registered: 2017-08-29
Posts: 58

xsibackup-rsync process goes to 100% cpu, transfer speed is almost 0

I noticed that after upgrading but i cannot remember which version, but seems that the binary wasn't changed. During a first onediff remote backup (don't know if it happens also on further backups) after a while the transfer speed reaches almost 0,5 kB/s and the process go to 100% cpu. After that sometimes it resumes with usual transfer. Could be the ssh2 forced option added recently? How can i debug this behaviour?

Using latest version, 10.1.0 on esxi 5.5


30,454,385,160  70%    6.44MB/s    0:31:33

34,058,371,072  79%   65.31kB/s   37:49:09

37,176,639,488  86%   50.70kB/s   31:37:48

37,178,601,872  86%    1.82kB/s  882:56:27

37,178,831,248  86%    0.64kB/s 2494:04:45

37,178,896,784  86%    0.64kB/s 2514:27:28

37,178,929,552  86%    0.61kB/s 2616:23:45

Offline

#2 2017-11-09 15:06:59

admin
Administrator
Registered: 2017-04-21
Posts: 1,030

Re: xsibackup-rsync process goes to 100% cpu, transfer speed is almost 0

Yes, most of the overhead in Rsync transfers is caused by the SSH encryption and/or compression. What kind of server/CPU are you using?

Offline

#3 2017-11-09 15:11:35

sistemi
Member
Registered: 2017-08-29
Posts: 58

Re: xsibackup-rsync process goes to 100% cpu, transfer speed is almost 0

admin wrote:

Yes, most of the overhead in Rsync transfers is caused by the SSH encryption and/or compression. What kind of server/CPU are you using?

Ok, but when the transfer speed is normal the cpu is about 40-50%.

Manufacturer:     Dell Inc.
Model:     PowerEdge R720xd
Processors:     8 CPU x 1,999 GHz
Processor Type:     Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz
Hyperthreading:     Active
Total Memory:     95,96 GB

Offline

#4 2017-11-09 15:16:34

admin
Administrator
Registered: 2017-04-21
Posts: 1,030

Re: xsibackup-rsync process goes to 100% cpu, transfer speed is almost 0

Mmm, that does not seem to be a problem with your CPU capabilities. We have not experienced this issue when using SSH 2.0 dynamic compression. We'll try to add a flag to disable it.

Offline

#5 2017-11-09 16:27:14

admin
Administrator
Registered: 2017-04-21
Posts: 1,030

Re: xsibackup-rsync process goes to 100% cpu, transfer speed is almost 0

Just disable compression in your /etc/ssh/ssh_config file to find out if this is the cause of your slow down.
Add the line to the SSH client config file.

Compression no

Offline

#6 2017-11-09 16:46:25

admin
Administrator
Registered: 2017-04-21
Posts: 1,030

Re: xsibackup-rsync process goes to 100% cpu, transfer speed is almost 0

Another approach would be to use compression, but set its level down to 1

CompressionLevel 1

This yields the best results on fast networks

EXPLANATION:
When you use compression on highly compressed data, you obtain the opposite result; data is not compressed at all, but the CPU time to process it slows the transfer down. So, compression is only effective when you have big sparse files with a lot of empty space per instance.  In any case SSH2 dynamic compresson algorithm is sopoused to be able to detect wether compression needs to be turned on and up to what point.

Pro version uses XSIDiff to copy data the first time, it knows how to skip zero blocks, so compression is not used here. In case of using Rsync with the free version, it still uses SSH 1.0

Offline

Board footer