©XSIBackup-Free: Free Backup Software for ©VMWare ©ESXi

Forum ©XSIBackup: ©VMWare ©ESXi Backup Software


You are not logged in.

#1 2021-05-15 09:30:26

herrep
Member
From: Munich
Registered: 2019-07-08
Posts: 77

XSIBackup-DC: tmp directories not removed after backup

Hi,

When running XSIBackup-DC with action --backup, I realize that sometimes temporary directories created below ./XSIBackup-DC/tmp remain there, although they have no content in it.

I cannot see any issues with permissions, as the XSIBackup-DC directory and tmp below have both permissions/owners drwxrwx--- root:root and are executed under root.

The last time, I got the following error messages in xsibackup.log:

rm: can't remove '/vmfs/volumes/nas/XSIBackup-DC/tmp/34486014/.vnfs-3d250a-.blocklog': Device or resource busy
rm: can't remove '/vmfs/volumes/nas/XSIBackup-DC/tmp/34486014': Directory not empty
2021-05-15T05:04:17 | Error code 1794 at file common.c, line 1794 | Error description: can't remove directory tree, error: 0

However, when checking the directory /vmfs/volumes/nas/XSIBackup-DC/tmp/34486014 afterwards, I can see that this directory is empty.

Probably worth to note is the following setup:

I run XSIBackup-DC with action --backup and a particular target directory for VMs to be backed up in hot state. During this run, no temp directory was left open. Everything fine. Immediately afterwards I run another XSIBackup-DC with action --backup and the same particular target directory for VMs to be backed up in warm state. During this run, the above errors were shown.

At present, I cannot exactly reconstruct under what conditions tmp directories left open.

Any hints?

Best regards,
Peter

Offline

#2 2021-05-15 10:26:57

admin
Administrator
Registered: 2017-04-21
Posts: 1,821

Re: XSIBackup-DC: tmp directories not removed after backup

We can always improve the source to try to handle unexpected situations better, that said: many of your issues point at some kind of sluggish behavior from part of your FS. We have had all kind of situations with users, xsibackup will not prevent you from doing something awkward, like backing up to some deduplicated FUSE FS, nonetheless that will not yield anything close to good results, as you would be producing such a load on the target FS that the mere fact that you could end some backup would be a miracle, yet not achieving any productive result.

The same happens when performing backups to VMFS6, we won't even comment on VMFS5, as it can handle just around 130,000 inodes, which is clearly insufficient to host any deduplicated repository.

VMFS6 can host millions of files (the number of inodes is not official, we have benchmarked it though), still VMFS6 has not been conceived to host millions of small files, but a few huge ones, which is right the opposite. Thus, you can't expect a good speed from it, in fact it's slow as hell when compared to some regular FS such as XFS or ext4. So, if you try to force VMFS6 to do something beyond its design principles, you might take it to its limits and start producing awkward results without even noticing you are doing something wrong.

We always take the time to explain this things in most posts covering backup targets.

Offline

#3 2021-05-16 12:04:48

herrep
Member
From: Munich
Registered: 2019-07-08
Posts: 77

Re: XSIBackup-DC: tmp directories not removed after backup

My backup system is a Synology NAS. The backup volume is of RAID type SHR with file system Btrfs. XSIBackup-DC runs on ESXi 6.7, while the backup volume is mounted on ESXi via NFS async mode.

Is there anything I could improve so that I get rid off this sluggish behavior?

In particular, the Synology NAS is the only backup host. Would you rather suggest to run XSIBackup-DC on Synology and have a remote connection to the ESXi to obtain the VM backup data?

Offline

#4 2021-05-16 18:00:37

admin
Administrator
Registered: 2017-04-21
Posts: 1,821

Re: XSIBackup-DC: tmp directories not removed after backup

We haven't tested (c)XSIBackup-DC on BTRFS, it's still under development, you should not use it in production, it will not be the best approach, in special if you have turned on deduplication.

(c)XSIBackup-DC has its own buil-in deduplication engine, you don't need additional deduplication. I believe many users don't realize what deduplicating data in real time with a small block size as BTRFS does means in terms of CPU and memory. If on top of that you add that Synology doesn't ussually have specially powerful CPUs nor much RAM..., you are creating the perfect storm.

Use XFS or ext4, as stated all over (c)XSIBackup-DC posts and manual.

(c)XSIBackup-DC has been designed to run in the (c)ESXi host. Running it in the Synology device is a terrible idea, please, don't do that.

Offline

#5 2021-05-16 18:06:47

herrep
Member
From: Munich
Registered: 2019-07-08
Posts: 77

Re: XSIBackup-DC: tmp directories not removed after backup

Having the Synology NAS in the LAN besides the ESXi, do you prefer xsibackup executed on the ESXi with the Synology NAS accessed over NFS?

Or do you rather suggest to go for the server/client mode with xsibackup running on ESXi having an IP-based target where xsibackup is additionally running on the Synology NAS?

Offline

#6 2021-05-16 18:15:56

admin
Administrator
Registered: 2017-04-21
Posts: 1,821

Re: XSIBackup-DC: tmp directories not removed after backup

Client/server will always be faster, as the CPU load is balanced between client and server. That will depend on if you are able to configure key authentication, which seems to be a problem with your DSM version default configuration.

Offline

Board footer