©XSIBackup-Free: Free Backup Software for ©VMWare ©ESXi

Forum ©XSIBackup: ©VMWare ©ESXi Backup Software


You are not logged in.

#1 2021-05-13 15:52:29

herrep
Member
From: Munich
Registered: 2019-07-08
Posts: 77

Impact on backup chain if sometimes the hash comparison fails

Hi,

My backup is based on backup-prog=onediff and further a deduplication via backup-prog=xsitools:z.

In rare cases I realize that single VMs report an error after having performed the hash comparison. While this may create some doubts whether the current backup of the VM is indeed usable, I assume that this failure does not have any impact on following onediff backups, as always the entire disks are stored.

Similarly, when trying to restore a backup from the deduplication (xsitools) exactly from the day with the failed hash comparison, it might be questionable if this works, but for any following backups this should not be an issue if there are no hash errors.

Am I right with these assumptions?

Best regards,
Peter

Offline

#2 2021-05-14 14:56:13

admin
Administrator
Registered: 2017-04-21
Posts: 1,821

Re: Impact on backup chain if sometimes the hash comparison fails

First of all you must know that (c)XSIBackup-Pro Classic has been deprecated, we only sell editions based in DC technology which is more advanced from all points of view.

You are making some wrong assumptions which in the end turn out to be half right and some right ones.

Wrong:

Onediff does not produce a full backup every time. It coalesces the differential data stored in an intermediate snapshot with the seed data available from previous backup cycles. Thus, if for some reason that differential snapshot fails at the time to be integrated with the previous seed data, you will end up with a wrong backup -flat.vmdk file.

To read: A post on silent corruption with extended information

In the end your assumption turns out to be half right, as most of the checksum errors are due to read errors, that would mean that the base data is indeed O.K.. On top of that they tend to be sticky, thus you may repeat a failed checksum and get the same result, which is double puzzling.

The proof of that is that you may receive a wrong checksum in one backup cycle, but still get a good one on some subsequent one. As the checksum is indeed done on the full length of the -flat.vmdk file, a valid checksum comparison means that your data is indeed the same on both sides. That could only be possible if the previous failed checksums were spurious.


Right:

The possibility that you have a coincidence in a hash by chance is despicable, thus we can assume that any block with a given hash represents the data there contained, with the exception of a silent write or compression error, which is also quite despicable, as there are a multitude of intermediate checks which would need to all fail for that to happen.

Per instance, let's say that you read a chunk of data get its checksum, compress it and send it over the wire. If you get no error, that means that the data was read without an error via a system call with its own integrity mechanisms, then a checksum was calculated for it, also with its own integrity mechanisms, then the chunk is sent over an SSH tunnel with integrity checks and the tunneled data is in turn eventually sent over TCP to a temp file which is renamed at the end of the transmission only if no error was returned.

You can assume your stored blocks are as OK as your disks are.

Thus, if you receive an error in some XSITools deduplicated backup and it is interrupted. The previous and subsequent backup cycles which do not return any errors will be OK, as will be compounded by error free blocks.

Nonetheless you may end up with some garbage, if some block in a broken backup is not used any more by any deduplicated set.

Offline

#3 2021-05-14 15:07:09

herrep
Member
From: Munich
Registered: 2019-07-08
Posts: 77

Re: Impact on backup chain if sometimes the hash comparison fails

Thanks a lot for this clarification.

I understood that corrupted differential data will indeed break the backup chain. But if he hash check flags an error and then works again on the next day, I can assume this to be a false negative warning so that everything is all right if the hash check does not flag an error the next day. But if the hash check remains negative, then the backup chain is broken so that the differential backup has to be started from zero again.

Offline

#4 2021-05-14 15:10:16

admin
Administrator
Registered: 2017-04-21
Posts: 1,821

Re: Impact on backup chain if sometimes the hash comparison fails

I was still writing when you answered, don't miss the rest of it.

Yes, that assumption is correct.

Most, if not all, software manufacturers don't offer a checksum comparison mechanism on such big files as virtual disks. The reason is that most people won't try to understand how things work, they will just call their support help desks and drive them crazy, assuming that a wrong checksum means the backup failed. Thus, it's probably not feasible to offer such characteristic to the great public, the same way that it would be pointless to try to explain people that things are literally 'not there' when nobody looks at them.

Offline

#5 2021-05-14 22:45:49

herrep
Member
From: Munich
Registered: 2019-07-08
Posts: 77

Re: Impact on backup chain if sometimes the hash comparison fails

Thank you very much for all these detailed explanations!

Offline

Board footer