#1 2020-06-30 11:24:10

sphen
Member
Registered: 2020-06-13
Posts: 33

SIGTERM (13) condition - Receiver can't evaluate ? as a local path

Intermittently I get SIGTERM (13) condition during a backup. This has been encountered across different hosts, in this case I am using 13.0.2 from ESXi 6.7 to Ubuntu 18 via ssh. If I re-run the backup task immediately it typically goes past the error. I have seen it a handful of times.

I have found the corresponding errors from the receiver end's error.log:

2020-06-26T02:12:56 | Error code 560 at file xsibackup.c, line 560
Error description: can't evaluate ?p?ュg as a local path
2020-06-26T11:24:54 | Error code 560 at file xsibackup.c, line 560
Error description: can't evaluate ?"? as a local path
Error description: can't evaluate )T
                                    _no as a local path
2020-06-30T11:10:15 | Error code 560 at file xsibackup.c, line 560
Error description: can't evaluate ? as a local path

Any ideas? Thanks.

Offline

#2 2020-06-30 12:17:48

sphen
Member
Registered: 2020-06-13
Posts: 33

Re: SIGTERM (13) condition - Receiver can't evaluate ? as a local path

command and output for ref

[root@supernaut:~] /vmfs/volumes/SSD2-Super/XSIBackup-DC/xsibackup --backup "VMs(Graylog)" root@192.168.246.77:22:/media/wesley --ssh-ciphers=aes128-c
tr --rotate=3 --verbosity=10
-----------------------------------------------------------------------------------------------------------
|---------------------------------------------------------------------------------|
||-------------------------------------------------------------------------------||
|||   (c)XSIBackup-DC 1.3.0.2: Backup & Replication Software                    |||
|||   (c)33HOPS, Sistemas de Informacion y Redes, S.L. | All Rights Reserved    |||
||-------------------------------------------------------------------------------||
|---------------------------------------------------------------------------------|
                   (c)Daniel J. Garcia Fidalgo | info@33hops.com
|---------------------------------------------------------------------------------|
System Information: ESXi, Kernel 6 Major 7 Minor 0 Patch 0
-----------------------------------------------------------------------------------------------------------
License: 000506630000000000000000ac1f6b1dc42e
-----------------------------------------------------------------------------------------------------------
Primary TMP folder: /tmp/xsi/2215205
-----------------------------------------------------------------------------------------------------------
Secondary TMP folder: /vmfs/volumes/SSD2-Super/XSIBackup-DC/tmp/2215205
-----------------------------------------------------------------------------------------------------------
SSH Out: tue jun 30 11:08:27 utc 2020
SSH: Tue Jun 30 11:08:28 UTC 2020
-----------------------------------------------------------------------------------------------------------
Remote system: linux
-----------------------------------------------------------------------------------------------------------
User defined ciphers: aes128-ctr
-----------------------------------------------------------------------------------------------------------
PID: 2215205, Running job as: root
-----------------------------------------------------------------------------------------------------------
LZJB compression has been enabled
-----------------------------------------------------------------------------------------------------------
Block size is 1048576 bytes (1024.00 KB)
-----------------------------------------------------------------------------------------------------------
Configured ciphers: aes128-ctr
-----------------------------------------------------------------------------------------------------------
Checking whether XSIBackup-DC can be found at </usr/bin/xsibackup>
-----------------------------------------------------------------------------------------------------------
XSIBackup-DC was found at </usr/bin/xsibackup>
-----------------------------------------------------------------------------------------------------------
Created remote folder at 192.168.246.77:22:/media/wesley/20200630110824
-----------------------------------------------------------------------------------------------------------
(c)XSIBackup-DC setting repository at /media/wesley
-----------------------------------------------------------------------------------------------------------
Found remote dir /media/wesley/data at 192.168.246.77:22
-----------------------------------------------------------------------------------------------------------
.xsitools file located at root@192.168.246.77:22:/media/wesley/.xsitools
Ordering blocks 100.00%
-----------------------------------------------------------------------------------------------------------
Hardware disk: /dev/disks/naa.600140562d158eedf443d41afd97afd8
-----------------------------------------------------------------------------------------------------------
Virtual Machine Name: Graylog
-----------------------------------------------------------------------------------------------------------
Creating snapshot VM : Graylog (powered on)
-----------------------------------------------------------------------------------------------------------
*** Snapshot was successfully created ***
-----------------------------------------------------------------------------------------------------------
Created remote folder at 192.168.246.77:22:/media/wesley/20200630110824/Graylog
-----------------------------------------------------------------------------------------------------------
Lock file: /media/wesley/20200630110824/Graylog/.locked
New backup/replica: Graylog | folder
-----------------------------------------------------------------------------------------------------------
Backup start date: 2020-06-30 11:08:42


....

Block 8371, exists: 0, sha1str: 328dbbc21effff1ee2feb95d264adf760a0ecf1c
::: detail :::  16.35% done | block    8372 out of   51200                               | Done  15.06%
Block 8372, exists: 0, sha1str: 3b11fb6c860c0dcfc1c4b54a01b11c9f61a68643
::: detail :::  16.35% done | block    8373 out of   51200                               | Done  15.06%
Block 8373, exists: 0, sha1str: 25b54727cb314bb77d4db9442fbd71b97c919553
::: detail :::  16.36% done | block    8374 out of   51200                               | Done  15.06%
Block 8374, exists: 0, sha1str: 8125e635276bcf3bc169dc20e7db96cb75e5a1bc
::: detail :::  16.36% done | block    8375 out of   51200                               | Done  15.06%
Block 8375, exists: 0, sha1str: bbf583a5cbd60ffd83db201fb2802339ce87de43

-----------------------------------------------------------------------------------------------------------

SIGTERM (13) condition was trapped: check logs for more details
-----------------------------------------------------------------------------------------------------------
Cleaning up...
-----------------------------------------------------------------------------------------------------------
*** Snapshot was removed ***
-----------------------------------------------------------------------------------------------------------
Removed <tmp> dir        OK
-----------------------------------------------------------------------------------------------------------
Unlocked backup          OK
-----------------------------------------------------------------------------------------------------------
Removed PID              OK
-----------------------------------------------------------------------------------------------------------

Offline

#3 2020-07-01 10:38:22

admin
Administrator
Registered: 2017-04-21
Posts: 1,279

Re: SIGTERM (13) condition - Receiver can't evaluate ? as a local path

Well, it's clear that the server is receiving file names which are ackward and they are breaking regexp path integrity rules on the server side, which is in turn causing the server to drop the tunnel.

You aren't showing the name of the files you are trying to backup, maybe you need some name + path cleaning.

Another possibility is that some packets are lost due to an unstable link and the server drops the connection when it receives some unformatted garbage. There are previous checks to path not matching though.

Offline

#4 2020-07-01 10:51:30

sphen
Member
Registered: 2020-06-13
Posts: 33

Re: SIGTERM (13) condition - Receiver can't evaluate ? as a local path

It is breaking on vmdk when it does. File paths are fine. the above example file is going from:

/vmfs/volumes/5b6105f0-96792252-f5ce-ac1f6b1dc5be/Graylog/Graylog.vmdk

to

root@192.168.246.77:22:/media/wesley

I would also not imagine network issue in this environment as there has been no indicated issue of it previously.  Either way what would be the best way to get more data when this failure occurs to help diagnose?

Thanks.

Offline

#5 2020-07-01 11:06:59

admin
Administrator
Registered: 2017-04-21
Posts: 1,279

Re: SIGTERM (13) condition - Receiver can't evaluate ? as a local path

If your file names and paths are orthodox in naming, then you are losing packets beyond what TCP can handle.

You may use WinMTR to have an insight into what might be going on:
https://sourceforge.net/projects/winmtr/files/

Or use some more classic tool such as ping and/ or tcpdump

A regular ping can offer some insight, pay attention to whether some packet is lost and also wht's the jitter, the difference between the largest and the lowest ping times.

You will need to perform the tests while you are backing up, sometimes it's a poorly written driver that could be causing the problems, nevertheless it could be any network element.

Offline

Board footer