Forum ©XSIBackup: ©VMWare ©ESXi Backup Software
MenuYou are not logged in.
- Topics: Active | Unanswered
Pages: 1
#1 2023-11-28 03:08:49
- nikiwaibel
- Member
- Registered: 2023-11-28
- Posts: 3
Segmentation fault
Hi *, XSIBackup-DC was running fine for about three months, then it started to produce "Segmentation fault". unfortunately, this was not recognized for quite a while :-/
output:
[root@localhost:/vmfs/volumes/4faaf6df-f7d771f1-0000-000000000000/XSIBackup-DC] /vmfs/volumes/BACKUP/XSIBackup-DC/xsibackup --backup=cbt "VMs(restore)" "/vmfs/volumes/BACKUP/ALL" --quiesce="" --description="server-2 and restore" --options="S" --verbosity=10 --debug-print
|---------------------------------------------------------------------------------|
||-------------------------------------------------------------------------------||
||| (c)XSIBackup-DC 1.5.1.17: Backup & Replication Software |||
||| (c)33HOPS, Sistemas de Informacion y Redes, S.L. | All Rights Reserved |||
||-------------------------------------------------------------------------------||
|---------------------------------------------------------------------------------|
(c)Daniel J. Garcia Fidalgo | info@33hops.com
|---------------------------------------------------------------------------------|
System Information: ESXi, Kernel 6 Major 7 Minor 0 Patch 0
-------------------------------------------------------------------------------------------------------------
License: 000206C20000000000000000e41f13b8ae90 | (c)XSIBackup-DC
-------------------------------------------------------------------------------------------------------------
Primary TMP folder: /tmp/xsi/2423890
-------------------------------------------------------------------------------------------------------------
Secondary TMP folder: /vmfs/volumes/BACKUP/XSIBackup-DC/tmp/2423890
-------------------------------------------------------------------------------------------------------------
Ciphers configured as autonegotiation mode
-------------------------------------------------------------------------------------------------------------
PID: 2423890, Running job as: root
-------------------------------------------------------------------------------------------------------------
LZJB compression has been enabled
-------------------------------------------------------------------------------------------------------------
Block size is 1.00 MB (1048576 bytes)
-------------------------------------------------------------------------------------------------------------
Performing --backup action
-------------------------------------------------------------------------------------------------------------
Created backup folder at: /vmfs/volumes/BACKUP/ALL/20231128030004
-------------------------------------------------------------------------------------------------------------
Backup folder '/vmfs/volumes/BACKUP/ALL/20231128030004'
-------------------------------------------------------------------------------------------------------------
(c)XSIBackup-DC setting repository at /vmfs/volumes/BACKUP/ALL
-------------------------------------------------------------------------------------------------------------
.xsitools file located: /vmfs/volumes/BACKUP/ALL/.xsitools
-------------------------------------------------------------------------------------------------------------
Repository info: bs is 1048576, compression is 1
-------------------------------------------------------------------------------------------------------------
Getting .blocklog data...
-------------------------------------------------------------------------------------------------------------
LCMD: R="$( (ls '/vmfs/volumes/BACKUP/ALL/data/.blocklog' 2>/dev/null | wc -l 2>/dev/null) 2>&1 )";echo -n $?";"$R
-------------------------------------------------------------------------------------------------------------
DEBUG PRINT: R="$( (ls '/vmfs/volumes/BACKUP/ALL/data/.blocklog' 2>/dev/null | wc -l 2>/dev/null) 2>&1 )";echo -n $?";"$R, OUTPUT: 0;1
-------------------------------------------------------------------------------------------------------------
Sorting local .blocklog data...
-------------------------------------------------------------------------------------------------------------
LCMD: sort -u '/vmfs/volumes/BACKUP/XSIBackup-DC/tmp/2423890/.blocklog' | awk -F ';' '{print $1}' > '/vmfs/volumes/BACKUP/XSIBackup-DC/tmp/2423890/.blocklog.7888164332087440' && mv -f '/vmfs/volumes/BACKUP/XSIBackup-DC/tmp/2423890/.blocklog.7888164332087440' '/vmfs/volumes/BACKUP/XSIBackup-DC/tmp/2423890/.blocklog' && echo 1 || echo 0
-------------------------------------------------------------------------------------------------------------
Mapping local .blocklog data...
-------------------------------------------------------------------------------------------------------------
Item number 1 in this job
-------------------------------------------------------------------------------------------------------------
Segmentation faultstrace:
write(1, "Mapping local .blocklog data...\n", 32Mapping local .blocklog data...
) = 32
write(1, "\33[90m---------------------------"..., 119-------------------------------------------------------------------------------------------------------------
) = 119
stat("/vmfs/volumes/BACKUP/XSIBackup-DC/tmp/2423806/.blocklog", {st_mode=S_IFREG|0644, st_size=403527330, ...}) = 0
open("/vmfs/volumes/BACKUP/XSIBackup-DC/tmp/2423806/.blocklog", O_RDONLY) = 5
mmap(NULL, 403527330, PROT_READ, MAP_PRIVATE, 5, 0) = 0x66a19f2000
close(5) = 0
mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x66b9ac8000
brk(0x66a19d2000) = 0x66a18b1000
mmap(NULL, 1183744, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x66ba2c9000
brk(0x66a19d2000) = 0x66a18b1000
mmap(NULL, 1183744, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x66ba3ea000
access("VMs(restore)", F_OK) = -1 ENOENT (No such file or directory)
stat("VMs(restore)", 0x3d8ca4b28c0) = -1 ENOENT (No such file or directory)
stat("VMs(restore)", 0x3d8ca4b28c0) = -1 ENOENT (No such file or directory)
write(1, "Item number 1 in this job\n", 26Item number 1 in this job
) = 26
write(1, "\33[90m---------------------------"..., 119-------------------------------------------------------------------------------------------------------------
) = 119
pipe([5, 6]) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x66a188f910) = -1 ENOSPC (No space left on device)
close(6) = 0
close(5) = 0
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
pipe([5, 6]) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x66a188f910) = -1 ENOSPC (No space left on device)
close(6) = 0
close(5) = 0
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
+++ killed by SIGSEGV +++
Segmentation faultLast edited by nikiwaibel (2023-11-28 03:16:05)
Offline
#2 2023-11-29 10:30:09
- admin
- Administrator
- Registered: 2017-04-21
- Posts: 2,055
Re: Segmentation fault
There is a bug we just fixed by which if some VM in the list of VMs to backup does not exist any more the job SEGFAULTS. Check that you aren't trying to backup some VM that you removed or renamed.
Also, by the way you describe your issue, you could have filled some temp folder. Check that you have space in the root FS and /tmp by issuing:
df -h
vdf -hWe recommend that you update to the latest version, yours is rather outdated.
Offline
#3 2023-12-13 05:25:14
- nikiwaibel
- Member
- Registered: 2023-11-28
- Posts: 3
Re: Segmentation fault
the VM mentioned in the job "VMs(restore)" exists, for sure:
[root@localhost:~] esxcli vm process list
SERVER01
World ID: 2099837
Process ID: 0
VMX Cartel ID: 2099826
UUID: 56 4d 89 9a e6 51 33 f6-b5 67 c7 6a 9f 0a d8 54
Display Name: SERVER01
Config File: /vmfs/volumes/5d69e461-d6473680-b727-e41f13b8ae90/SERVER01/SERVER01.vmx
server-2
World ID: 2100029
Process ID: 0
VMX Cartel ID: 2100028
UUID: 56 4d f1 85 36 86 d2 96-bb 45 41 9b 5e aa 94 8c
Display Name: server-2
Config File: /vmfs/volumes/5d69e461-d6473680-b727-e41f13b8ae90/server-2/server-2.vmx
restore
World ID: 2100263
Process ID: 0
VMX Cartel ID: 2100262
UUID: 56 4d 72 73 27 4a 70 ff-0d ef 1c e6 09 2e 2d a3
Display Name: restore
Config File: /vmfs/volumes/62c2431a-e1996108-7496-e41f13b8ae90/restore/restore.vmx
[root@localhost:~]It also seems, there is enough space:
[root@localhost:~] df -h
Filesystem Size Used Available Use% Mounted on
NFS41 19.9T 9.5T 10.4T 48% /vmfs/volumes/BACKUP
VMFS-6 7.3T 3.8T 3.4T 53% /vmfs/volumes/DS-1
VMFS-6 278.2G 9.2G 269.1G 3% /vmfs/volumes/DS-2
VMFS-6 8.0T 468.0G 7.5T 6% /vmfs/volumes/DATASTORE
vfat 285.8M 183.5M 102.3M 64% /vmfs/volumes/65365cda-2623dfaa-5fd7-e41f13b8ae90
vfat 249.7M 153.0M 96.7M 61% /vmfs/volumes/2d4a6ae6-df1abc5e-8b1c-471caacbeb48
vfat 249.7M 151.0M 98.7M 60% /vmfs/volumes/bdeb22e2-6c4aa262-47f6-d7ca8405d568
[root@localhost:~] vdf -h
Tardisk Space Used
vmx.v00 103M 103M
vim.v00 112M 112M
sb.v00 189M 189M
s.v00 59M 59M
ata_liba.v00 296K 294K
ata_pata.v00 48K 44K
ata_pata.v01 36K 35K
ata_pata.v02 40K 36K
ata_pata.v03 40K 37K
ata_pata.v04 44K 41K
ata_pata.v05 40K 38K
ata_pata.v06 36K 35K
ata_pata.v07 40K 39K
block_cc.v00 88K 84K
bnxtnet.v00 560K 557K
bnxtroce.v00 304K 303K
brcmfcoe.v00 2M 2M
char_ran.v00 52K 51K
ehci_ehc.v00 96K 95K
elxiscsi.v00 516K 514K
elxnet.v00 644K 641K
hid_hid.v00 64K 60K
i40en.v00 632K 631K
iavmd.v00 180K 176K
igbn.v00 332K 329K
ima_qla4.v00 1M 1M
ipmi_ipm.v00 44K 40K
ipmi_ipm.v01 84K 82K
ipmi_ipm.v02 104K 102K
iser.v00 236K 234K
ixgben.v00 536K 532K
lpfc.v00 2M 2M
lpnic.v00 644K 641K
lsi_mr3.v00 368K 366K
lsi_msgp.v00 516K 515K
lsi_msgp.v01 560K 560K
lsi_msgp.v02 548K 544K
misc_cni.v00 24K 22K
misc_dri.v00 1M 1M
mtip32xx.v00 256K 254K
ne1000.v00 640K 639K
nenic.v00 272K 270K
net_bnx2.v00 288K 286K
net_bnx2.v01 2M 2M
net_cdc_.v00 28K 25K
net_cnic.v00 148K 145K
net_e100.v00 312K 311K
net_e100.v01 356K 353K
net_enic.v00 176K 175K
net_fcoe.v00 80K 79K
net_forc.v00 128K 125K
net_igb.v00 324K 321K
net_ixgb.v00 412K 409K
net_libf.v00 76K 75K
net_mlx4.v00 356K 355K
net_mlx4.v01 240K 236K
net_nx_n.v00 1M 1M
net_tg3.v00 316K 312K
net_usbn.v00 56K 53K
net_vmxn.v00 112K 108K
nfnic.v00 548K 544K
nhpsa.v00 568K 565K
nmlx4_co.v00 728K 726K
nmlx4_en.v00 756K 755K
nmlx4_rd.v00 328K 326K
nmlx5_co.v00 1M 1M
nmlx5_rd.v00 260K 257K
ntg3.v00 124K 121K
nvme.v00 296K 293K
nvmxnet3.v00 172K 171K
nvmxnet3.v01 172K 170K
ohci_usb.v00 64K 63K
pvscsi.v00 120K 118K
qcnic.v00 284K 281K
qedentv.v00 2M 2M
qfle3.v00 2M 2M
qfle3f.v00 1M 1M
qfle3i.v00 336K 332K
qflge.v00 504K 500K
sata_ahc.v00 88K 87K
sata_ata.v00 60K 56K
sata_sat.v00 68K 66K
sata_sat.v01 48K 46K
sata_sat.v02 52K 48K
sata_sat.v03 40K 39K
sata_sat.v04 36K 34K
scsi_aac.v00 180K 178K
scsi_adp.v00 452K 450K
scsi_aic.v00 296K 292K
scsi_bnx.v00 284K 280K
scsi_bnx.v01 204K 203K
scsi_fni.v00 244K 242K
scsi_hps.v00 212K 211K
scsi_ips.v00 108K 106K
scsi_isc.v00 44K 42K
scsi_lib.v00 212K 211K
scsi_meg.v00 100K 99K
scsi_meg.v01 176K 173K
scsi_meg.v02 96K 93K
scsi_mpt.v00 460K 459K
scsi_mpt.v01 504K 501K
scsi_mpt.v02 432K 429K
scsi_qla.v00 292K 289K
sfvmk.v00 604K 602K
shim_isc.v00 16K 15K
shim_isc.v01 16K 15K
shim_lib.v00 44K 43K
shim_lib.v01 44K 43K
shim_lib.v02 24K 20K
shim_lib.v03 24K 20K
shim_lib.v04 12K 9K
shim_lib.v05 12K 9K
shim_vmk.v00 220K 217K
shim_vmk.v01 232K 228K
shim_vmk.v02 236K 232K
smartpqi.v00 284K 281K
uhci_usb.v00 64K 63K
usb_stor.v00 164K 162K
usbcore_.v00 320K 318K
vmkata.v00 204K 201K
vmkfcoe.v00 936K 935K
vmkplexe.v00 48K 46K
vmkusb.v00 988K 986K
vmw_ahci.v00 240K 236K
xhci_xhc.v00 236K 235K
elx_esx_.v00 4M 4M
btldr.t00 588K 584K
esx_dvfi.v00 492K 488K
esx_ui.v00 21M 21M
esxupdt.v00 576K 572K
weaselin.t00 3M 3M
lsu_hp_h.v00 152K 149K
lsu_inte.v00 44K 41K
lsu_lsi_.v00 420K 417K
lsu_lsi_.v01 240K 237K
lsu_lsi_.v02 496K 493K
lsu_lsi_.v03 264K 260K
lsu_lsi_.v04 560K 557K
lsu_smar.v00 144K 141K
native_m.v00 820K 818K
qlnative.v00 2M 2M
rste.v00 828K 825K
vmware_e.v00 188K 186K
vsan.v00 43M 43M
vsanheal.v00 7M 7M
vsanmgmt.v00 25M 25M
xorg.v00 3M 3M
imgdb.tgz 1M 1M
state.tgz 20K 18K
vmware_s.v00 5M 5M
-----
Ramdisk Size Used Available Use% Mounted on
root 32M 2M 29M 7% --
etc 28M 268K 27M 0% --
opt 32M 144K 31M 0% --
var 48M 2M 45M 5% --
tmp 256M 12K 255M 0% --
iofilters 32M 0B 32M 0% --
shm 1024M 0B 1024M 0% --
hostdstats 803M 3M 799M 0% --
[root@localhost:~]latest version i've to check, as Your support finished on 2023-07-23T00:00:00+00:00). unfortunately the backup was working for about 2 months only (2022-07-16…2022-09-30), then it created the backup-dirs (it looked ok) but without content (that is why it was not recognized earlier).
i've already deleted all empty dirs in the meantime, except for tests i did recently (since 2023-11-28)
[root@localhost:~] ls /BACKUP/ALL/*
/BACKUP/ALL/20220716040201:
server-2
/BACKUP/ALL/20220716201900:
server-2
/BACKUP/ALL/20220716204914:
server-2
/BACKUP/ALL/20220717044701:
SERVER01
/BACKUP/ALL/20220717051300:
SERVER01
/BACKUP/ALL/20220723094000:
SERVER01 restore server-2
/BACKUP/ALL/20220723122200:
SERVER01
[--------------cut--------------]
/BACKUP/ALL/20220928202200:
restore server-2
/BACKUP/ALL/20220928212300:
SERVER01
/BACKUP/ALL/20220929202200:
restore server-2
/BACKUP/ALL/20220929212300:
SERVER01
/BACKUP/ALL/20220930202200:
restore server-2
/BACKUP/ALL/20231128013848:
/BACKUP/ALL/20231128015705:
/BACKUP/ALL/20231128020237:
/BACKUP/ALL/20231128020407:
/BACKUP/ALL/20231128020457:
/BACKUP/ALL/20231128020611:Offline
#4 2023-12-18 20:27:05
- nikiwaibel
- Member
- Registered: 2023-11-28
- Posts: 3
Re: Segmentation fault
version 1.7.1.3 works as expected. same configuration, no segfaults.
just for reference, the esxi host is a fully patched 6.7u3: ESXi-6.7.0-20221004001-standard aka build#20497097 aka ESXi 6.7 P08
Offline
Pages: 1