Getting "backup failed error" but "OK" on all in the status.

nle · 2017-08-28 09:07:21

Hi,

the backup is running on VMware ESXi 6.5.0 build-5969303, copying to a NFS datastore.

I'm getting this subject:

✖ XSIBACKUP errors detected in backup job!! | ESXi Backup Report

But in the e-mail it says "ok" on all VMs.

Done hot backup, using vmkfstools.
The backup room has been limited to 200 Gb.
Available room in device /vmfs/volumes/XXX/ESXi/VM_Backup before backup: 200 Gb.
Needed room in device /vmfs/volumes/XXX/ESXi/VM_Backup for backup: 81 Gb.
VM Name			State	Size	Stop		Copy	Start	Time (min)	Speed (mb/s)
XXX (vMA)		ON	1G	NO (hot backup)	OK	-	1		16
XXX (Ubuntu BE)		ON	32G	NO (hot backup)	OK	-	20		26
XXX (Ubuntu LTS)	ON	40G	NO (hot backup)	OK	-	16		41
XXX (Ubuntu BE)		ON	7G	NO (hot backup)	OK	-	8		15
Available space in device /vmfs/volumes/XXX/ESXi/VM_Backup after backup: 200 Gb.
Complete backup elapsed time: 47 min

Any advice on this?

Last edited by nle (2017-08-28 09:09:01)

admin · 2017-08-28 12:12:06

Check the full log to dive into the issue. Look for any line containing the word error

cat xsibackup-cron.log | grep -i error

Use tail, per instance, to get errors from the last N (200 in the example) lines

tail -n200 xsibackup-cron.log | grep -i error

nle · 2017-08-28 14:53:20

Thank you!

Should have thought about that, but unfortunately, it does not give me much more to go on.

Destination disk format: VMFS thin-provisioned
Cloning disk '/vmfs/volumes/VMs/XXX/XXX_0.vmdk'...
Clone: 100% done.
Error backing the ESXi host config to /vmfs/volumes/XXX/ESXi/VM_Backup
---------------------------------------------------------------------------------------------------------------------------------
2017-08-28T06:48:03|  Errors detected in backup, check logs
---------------------------------------------------------------------------------------------------------------------------------
2017-08-28T06:48:03|  Some warnings were raised

Why the error copying the config file? What warnings? All I see is errors.

Last edited by nle (2017-08-28 14:53:36)

admin · 2017-08-28 18:37:00

Well, this is clearly the error causing your behaviour:

Error backing the ESXi host config to /vmfs/volumes/XXX/ESXi/VM_Backup

And that explains why the individual VM backups are O.K.
Run this commands manually to determine why it's returning an error:

vim-cmd hostsvc/firmware/backup_config
find /scratch/downloads -type f -name "*.tgz" -exec ls -1rt "{}" + | head -n1

nle · 2017-08-28 19:33:02

Thank you.

Yes that is the error, but it says some warnings were raised – but I can't seem to locate any warnings. But I guess it's a general message if there is an error?

Command outputs:

[root@hostname:~] vim-cmd hostsvc/firmware/backup_config
(vmodl.fault.SystemError) {
   faultCause = (vmodl.MethodFault) null,
   faultMessage = <unset>,
   reason = "Internal error"
   msg = "Received SOAP response fault from [<cs p:047856a8, TCP:localhost:80>]: backupConfiguration
A general system error occurred: Internal error"
}

[root@hostname:~] find /scratch/downloads -type f -name "*.tgz" -exec ls -1rt "{}" + | head -n1
find: /scratch/downloads: No such file or directory

The commands are run from default directory "/", and "/scratch/downloads" does not exist (scratch does exist, but does not have downloads dir).

Edit found [url=https://communities.vmware.com/thread/567954]this post[/url] on the VMWare forums.

Edit 2: Seems like creating the "downloads" directory and uninstalling the iscsi driver things did the trick.

Last edited by nle (2017-08-28 20:19:26)

admin · 2017-08-29 15:36:55

Great!. ESXi is far from perfection. It sometimes fails at performing the simplest things.
The downloads dir should be created automatically.
We'll improve this piece of code to double check this fact.

nle · 2017-09-18 09:01:31

Update. I had to restart the server, and it did not create "/scratch/downloads" folder (or leave the existing be) upon restart. So the backup task failed again (set up to execute every Sunday night).

At least I now know the reason, I'll try to see if I can make the folder persistent.

admin · 2017-09-18 16:54:04

When you run this command

vim-cmd hostsvc/firmware/backup_config

The [b].tgz[/b] file is saved to [b]/scratch/downloads[/b] by default.
If you have some manufacturer customized ESXi distro, or some other sort of custom one, you might need to edit the [b]src/esxbackup[/b] file to adapt it to your distro.

nle · 2017-09-18 18:46:24

Thanks.

It's a custom dell ESXi image, but I think this is a bug – not a feature/thingie on the Dell image. Especially since it's mentioned in the VMWare community.

admin · 2017-09-19 02:54:27

We have found custom images to be pretty buggy, not only this brand but some others too. We recommend that you stick to official VMWare releases.

Forum ©XSIBackup: ©VMWare ©ESXi Backup Software

#1 2017-08-28 09:07:21

Getting "backup failed error" but "OK" on all in the status.

#2 2017-08-28 12:12:06

Re: Getting "backup failed error" but "OK" on all in the status.

#3 2017-08-28 14:53:20

Re: Getting "backup failed error" but "OK" on all in the status.

#4 2017-08-28 18:37:00

Re: Getting "backup failed error" but "OK" on all in the status.

#5 2017-08-28 19:33:02

Re: Getting "backup failed error" but "OK" on all in the status.

#6 2017-08-29 15:36:55

Re: Getting "backup failed error" but "OK" on all in the status.

#7 2017-09-18 09:01:31

Re: Getting "backup failed error" but "OK" on all in the status.

#8 2017-09-18 16:54:04

Re: Getting "backup failed error" but "OK" on all in the status.

#9 2017-09-18 18:46:24

Re: Getting "backup failed error" but "OK" on all in the status.

#10 2017-09-19 02:54:27

Re: Getting "backup failed error" but "OK" on all in the status.

Board footer