Forum ©XSIBackup: ©VMWare ©ESXi Backup Software

We had not fully understood what you were trying to accomplish.

The replication process relies on the target disks having not changed since the last replication cycle, that's obviously a basic and unavoidable requirement, as otherwise you could corrupt your data.

The way (c)XSIBackup follows track of the disks states is by comparing their CIDs in the .vmdk file descriptors. If you do anything that alters this, (c)XSIBackup will asume a change in the target VM and resync from scratch.

Your output is plagued with an error message stating that some VM registered to your host is simply not there.

Skipping invalid VM '26'

We can't help you to detect your issue when you haven't even paid attention to that "little detail". For a host to be backed up, first of all it must be in a good working condition and under the control of the Sysadmin.

(c)XSIBackup relies on the output from part of the shell to detect errors and act accordingly, it can't just work with a recurring error message constantly bubbling up the shell.

Option --timestamp does not exist in (c)XSIBackup-DC
You can produce the same result by using something like the job below.

./xsibackup --replica "VMs(VM1,VM2)" "root@11.2.3.4:22:/vmfs/volumes/datastore1/replicas/$(date +%Y%m%d%H%M%S)

That will produce new replicas, each one of them in its own timestamped folder. From version 1.5.0.11 you will be able to rotate such structure of folders, still that method is rudimentary, as it implies copying all data every time.

It is indeed far less advanced than the previously proposed with a fixed number of folders set in the very same job design. If you use the code below, you don't need rotating and on top of that all replicas will be differential from the second pass

/scratch/XSI/XSIBackup-DC/xsibackup \
--replica[=cbt] \
"VMs(VM1,VM2, VM3...)" \
/vmfs/volumes/backup/replica/$(( $(date +%j) % 5 )) \
--use-smtp="1" \
--mail-to=mail-from \
>> /scratch/XSI/XSIBackup-DC/var/log/xsibackup.log 2>&1

Thank you for the feedback, it was clear to us that something in that VM layout was not right.

As said, as per version 1.5.0.10 you can't prune over IP (that will change in some future version), do it locally if you will, or even better, rotate your backups with a dynamic target:

./xsibackup --backup "VMs(VM1,VM2)" root@1.2.3.4:22:/path/to/your/backup/repo$(date +%m)

In the above example a new repository is created each month. Then archive (the corporate standard action) or delete the older repositories as the new one accumulates new data.

If you prefer to keep a single repository and prune the data, do it from the Linux VM. Still you will add the network latency of the Samba mount to each block deletion system call, which will multiply the time taken to achieve the effective deletion of the blocks. This is not much different to deleting a lot of small files from a share or from the same local FS, the difference in time will be substantial.

On the other side CIFS is not a very effective nor professional file share protocol for virtual machines, in special when compared to NFS or iSCSI. Try to mount the share as NFS instead of CIFS and format the drive or volume as XFS or ext4.

There's an even better approach, which is to add the drive as a [b][url=https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.storage.doc/GUID-B3522FF1-76FF-419D-8DB6-F15BFD4DF12A.html]raw device mapping[/url][/b], then format it as ext4 or XFS. By using a local disk you will get rid of the network latency associated to the file share protocol.

We know SMEs are limited in resources when compared to bigger enterprises, nonetheless, you can buy a 4TB Ironwolf Seagate HD for under 90.00USD nowadays, there's no excuse in terms of storage price.

These are just some ideas to take into practice without adding anything new or expensive to your system.

It could be a bug, we'll request that to be revised and eventually fixed in some next release.

Please post your job, otherwise we have to use our imagination to try to figure out what you mean.
Is the remote VM accessed via IP?
Is it mounted via NFS?

What do you mean by CIFS?.
VMWare (c)ESXi does not support CIFS.

If what you mean by a remote box that mounts CIFS is that you are using a remotely mounted CIFS share exposed though NFS or directly accessing it via IP, why would you want to do such thing?. It's OK if you want to experiment, still:

Why adding double latency?
Why using a file share protocol which is not recommended by us, nor by VMWare?
Did you at least fully try your setup?, did you restore some VM?

(c)XSIBackup is a tool, it's not user software, it will not keep you from doing anything, that doesn't mean that you should do anything. What we mean by that is that you may very well use it to store your backups in a USB stick attached to your Smart TV, still, although it may be a nice way to experiment and learn, we would never recommend that anybody uses that kind of approach to store their VM backups.

You are a Sysadmin, you are also creating when you design your systems, and with creative power comes responsibility. Knowing which pieces you can put together and how will they play along is something on your behalf.

The setup that we believe you are describing is not such an extreme case when compared to the USB stick + Smart TV example, still it's not very recommendable and we can't guarantee that it will work. As per the returned error it looks like the CIFS FS is not fully accessible to some (c)XSIBackup launched system call. You may have some finer details by prepending the command with 'strace'.

Apart from that, the --prune action is local, it can't be run directly from a command line to a remote file system this way except when used from a rotate argument.

./xsibackup --prune root@a.b.c.d:22:/some/remote/folder

Therefore, you might be committing some conceptual error as well. We don't know as we don't have all the details though.

Pruning is a destructive action, don't get us wrong, we use it every day for our own data. Still, there exist concerns on whether using it and how to use it in a corporate environment built on recommended tools and FSs for multiple reasons: [url=https://33hops.com/xsibackup-datacenter-pruning-old-backups.html]Concerns When Using Pruning on Corporate Backup Data Sets[/url].

If on top of the natural concerns around pruning data in a deduplicated repository you add some uncertainty of this caliber, you are simply being the first to cross on really thin ice.

You can't use /dest-dir as a valid backup path within (c)ESXi.
Please run something like

xsibackup --backup "VMs(vm-name)" /vmfs/volumes/backup/dest-dir >> logfile

And let us know if it worked.
You haven't still stated which version you are using.

WD MYCloud is not among the professional kind of NAS device you should expect close to theoretical limit speeds, still you should expect 20-30 MB/s of sustained throughput, especially if it's not a MyCloud NAS from the home line of appliances.

Your i7 3930K CPU is from 2011, still not bad, you should expect some decent performance from it.

Saying 1 gb NIC is close to saying nothing. Cheap devices, like the popular Realtek chipsets will not yield more than some MB/s when used in ESXi. Use Intel NICs, even cheap desktop ones will do it at good average rates.

Check your switch, it might be the culprit. You should not expect more than 15-20MB/s of effective troughput from cheap devices.

We don't know what's the state of your data, nonetheless what's important here is the -flat.vmdk files and the snapshots containing intermediate data. Try to not backup VMs with previous snapshots to simplify things. Also, if you know the topolgy of a chain of snapshots, which is rather simple, you can consolidate from any snapshot by using vmkfstools clone features too and generate a new consolidated VM.

Your hardware is clearly underperforming, please do make sure that you have activated the raid controller cache, especially if you own an HP server, 70-80%/20-30% R/W will do it.

Our software backs your VMs up. We don't know what snapshots you have or what time they were taken at. The backup snapshot is taken to backup the VM and then deleted once the backup finishes. If you abruptly interrupt some backup/replica process (c)XSIBackup will still be able to detect it and delete the backup snapshot automatically.

If for whatever reason you receive some kind of unhandled segfault, then you will have to delete the backup snapshot manually. It is clearly marked as XSINNNNNN..., so there isn't any possibility that you take it for something else.

Files are copied, but there isn't any kind of magical mechanism that can teleport to some other previous state of the VM apart from your own user snapshots.

Why they were taken or what they are doing there is something we can't get into.

The config backups are placed in a folder in the backup repository or in the replicated VM.
What do you mean by buried?, the config backup is easily accessible.
If you just want to backup the (c)ESXi host config, script and .tar the /etc folder that's what a config backup is in the end.

We fixed this in [url=https://33hops.com/xsibackup-copia-de-seguridad-vmware-esxi.html#prodchart]1.5.0.10[/url], already available to download. It was just a matter of tweaking an if clause, thus we released a quick fix.
Please confirm that you can now backup the VM as desired.