#1 Re: General matters » XSIBackup-DC: run --replica and make --backup from replicated VM » Today 10:33:50

There isn't any drawback, in fact it's our recommended procedure to backup big production VMs.

(c)XSIBackup 1.5.0.0 and above offers CBT compatibility for replicas, thus you can use the same approach that you used with Onediff, yet with a more advanced algorithm, as Onediff depended on a permanent backup snapshot on the source VM.

As with Onediff, you depend on the first part of the scheme working right.

CBT feature offers the option --options=R, which will create a test snapshot after each CBT round and will allow you to turn on the replicated VM without affecting the CBT replica.

./xsibackup --replica=cbt "VMs(WS2012)" root@192.168.3.83:22:/vmfs/volumes/datastore1 --options=R

#2 Re: General matters » XSIBackup-DC: tmp directories not removed after backup » Today 10:26:57

We can always improve the source to try to handle unexpected situations better, that said: many of your issues point at some kind of sluggish behavior from part of your FS. We have had all kind of situations with users, xsibackup will not prevent you from doing something awkward, like backing up to some deduplicated FUSE FS, nonetheless that will not yield anything close to good results, as you would be producing such a load on the target FS that the mere fact that you could end some backup would be a miracle, yet not achieving any productive result.

The same happens when performing backups to VMFS6, we won't even comment on VMFS5, as it can handle just around 130,000 inodes, which is clearly insufficient to host any deduplicated repository.

VMFS6 can host millions of files (the number of inodes is not official, we have benchmarked it though), still VMFS6 has not been conceived to host millions of small files, but a few huge ones, which is right the opposite. Thus, you can't expect a good speed from it, in fact it's slow as hell when compared to some regular FS such as XFS or ext4. So, if you try to force VMFS6 to do something beyond its design principles, you might take it to its limits and start producing awkward results without even noticing you are doing something wrong.

We always take the time to explain this things in most posts covering backup targets.

#3 Re: General matters » XSI DC 1.4.3.17: job with --replica --config-backup half works » Today 10:15:28

If you can steadily reproduce your error, you should first prepend strace to your job, run it and post at least the end of the output.

strace ./xsibackup --backup ....

#4 Re: General matters » XSIBackup-DC: No email report sent when xsibackup detects an error » Today 10:08:16

Don't use the --backup-how option if you want to perform hot backups, it's the default behavior.
11 is the received signal number, that is: a SEGFAULT. Which is caused when trying to access a restricted area of memory. It seems that due to the failed snapshot that condition is generated in your case. We should delve into it to at least handle the error in a nicer way and show some hint along with the error.

We have two general design principles considerations that are colliding and entering a contradiction there, which are: always trying to continue, no matter what happens to, at least, copy as much data as possible, with: control errors and show nice and explanatory responses. We should probably just continue to the next VM in case of some errors like a failed snapshot, as trying to continue with the backup process will just lead to undefined behavior.

The e-mail report is not sent due to the program jumping to the uncontrolled error handling function, namely: the program exists abruptly. We will add e-mail reporting to it in future versions.

#5 Re: General matters » XSI DC 1.4.3.17: job with --replica --config-backup half works » Today 09:52:08

As stated, we are talking about a simple atomic mkdir system call, not any complicated set of commands. Strace will offer you all the details on the issue. It could be that some DS in async mode would not create the folder in time for the subsequent commands to be able to copy to it. If that was the case, switching to Sync mode would solve the issue.

We could add a helper to wait until we detect the folder, but that would be misleading, as the rest of the program works in Sync mode. Could be some other thing, we'll keep investigating, we haven't been able to reproduce the issue though.

#6 Re: General matters » XSI DC 1.4.3.17: job with --replica --config-backup half works » Yesterday 18:18:41

This issue is not something obscure. It's a simple system call (mkdir) that creates that dir. You can run some job with strace (just prepend 'strace' to it) to detect the exact reason why the system call fails. It will produce a lot of output though, and will be slower.

Are you using some kind of Async option maybe?

#7 Re: General matters » VM replication completes with segmentation fault » Yesterday 18:08:35

It's not something about the data files, but some variable which is expected to contain some value but does not or is overflown.
It could be something as simple as some string length being overflown, some unexpected value in some field of any of the VM config files, etc..
This is not a generalized issue, thus, there must be something in your job/ environment triggering that behavior.

Provide as much feedback as you can. Start by posting all relevant information, that consists of the following:

1 - The full job that caused the issue.
2 - The full output
3 - Any relevant information, such as some unusual length in some path, email string, etc...

#8 Re: General matters » Binary regexfilt missing from XSIBackup-Free 1.4.3.18? » Yesterday 18:02:27

(c)XSIBackup-DC 1.5.0.3 has been published and the REGEXP simulator feature of the --exclude argument has been fixed.

#9 Re: General matters » Binary regexfilt missing from XSIBackup-Free 1.4.3.18? » Yesterday 17:36:12

There's still an issue with that REGEXP exclusion part of the GUI. It does not work.
It will be fixed in 1.5.0.3 later today.

It is the helper to simulate the disks/files that will be excluded that fails, not the exclude mechanism itself.
You can easily add that in the job file and test it in the command line.

#10 Re: © OneDiff » Impact on backup chain if sometimes the hash comparison fails » Yesterday 15:10:16

I was still writing when you answered, don't miss the rest of it.

Yes, that assumption is correct.

Most, if not all, software manufacturers don't offer a checksum comparison mechanism on such big files as virtual disks. The reason is that most people won't try to understand how things work, they will just call their support help desks and drive them crazy, assuming that a wrong checksum means the backup failed. Thus, it's probably not feasible to offer such characteristic to the great public, the same way that it would be pointless to try to explain people that things are literally 'not there' when nobody looks at them.

#11 Re: © OneDiff » Impact on backup chain if sometimes the hash comparison fails » Yesterday 14:56:13

First of all you must know that (c)XSIBackup-Pro Classic has been deprecated, we only sell editions based in DC technology which is more advanced from all points of view.

You are making some wrong assumptions which in the end turn out to be half right and some right ones.

Wrong:

Onediff does not produce a full backup every time. It coalesces the differential data stored in an intermediate snapshot with the seed data available from previous backup cycles. Thus, if for some reason that differential snapshot fails at the time to be integrated with the previous seed data, you will end up with a wrong backup -flat.vmdk file.

To read: A post on silent corruption with extended information

In the end your assumption turns out to be half right, as most of the checksum errors are due to read errors, that would mean that the base data is indeed O.K.. On top of that they tend to be sticky, thus you may repeat a failed checksum and get the same result, which is double puzzling.

The proof of that is that you may receive a wrong checksum in one backup cycle, but still get a good one on some subsequent one. As the checksum is indeed done on the full length of the -flat.vmdk file, a valid checksum comparison means that your data is indeed the same on both sides. That could only be possible if the previous failed checksums were spurious.


Right:

The possibility that you have a coincidence in a hash by chance is despicable, thus we can assume that any block with a given hash represents the data there contained, with the exception of a silent write or compression error, which is also quite despicable, as there are a multitude of intermediate checks which would need to all fail for that to happen.

Per instance, let's say that you read a chunk of data get its checksum, compress it and send it over the wire. If you get no error, that means that the data was read without an error via a system call with its own integrity mechanisms, then a checksum was calculated for it, also with its own integrity mechanisms, then the chunk is sent over an SSH tunnel with integrity checks and the tunneled data is in turn eventually sent over TCP to a temp file which is renamed at the end of the transmission only if no error was returned.

You can assume your stored blocks are as OK as your disks are.

Thus, if you receive an error in some XSITools deduplicated backup and it is interrupted. The previous and subsequent backup cycles which do not return any errors will be OK, as will be compounded by error free blocks.

Nonetheless you may end up with some garbage, if some block in a broken backup is not used any more by any deduplicated set.

#12 Re: General matters » XSIBackup-DC --replica: Keep the latest n versions of replicated VMs » Yesterday 14:41:13

You are probably assuming that DC works exactly the same way as Pro Classic, which is incorrect.
DC works by two main actions: --replica & --backup.

Replica produces replicas, it's not been thought to produce full backups, as replicas (and also backups) are differential and  if you create a new replica every time, you won't be taking advantage of any delta algorithm.

In any case if you still wish to do so, it's very easy, just add some dynamic replica target such as:

./xsibackup --replica "VMs(MyVM)" /vmfs/volumes/backups/replicas/$(date +%Y%m%d)

#13 Re: General matters » XSIBackup-DC: SMTP error 550 when using option --subject » Yesterday 14:35:17

You are receiving that error from the SMTP server, you probably added some text that triggered that response.

#14 Re: General matters » XSIBackup-DC --backup: Use of --rotate in a deduplicated backup repo » Yesterday 14:33:18

--rotate in a deduplicated repository is equivalent to pruning as much backups as to satisfy the rotation figure. Use to your discretion.

#15 Re: General matters » XSIBackup-DC: dir cfgbak only created with --backup and not --replica » Yesterday 14:31:27

We'll revise that behavior, by now, just create it manually before the replica.

#16 Re: General matters » How to keep multiple VMs powered off until cold backup is finished » Yesterday 14:28:38

You would need to script that, e.g.: turn off all VMs to backup, then turn them on when you are done.

#17 Re: General matters » Binary regexfilt missing from XSIBackup-Free 1.4.3.18? » Yesterday 14:27:19

Yes, that was probably a mistake at the time to pack latest versions. We have re-added the binary regexfilt to 1.5.0.2, just published.

#18 Re: General matters » Migrate from XSIBackup-Pro to XSIBackup-DC: Continue onediff/xsitools » Yesterday 13:53:49

No you can't, XSITools repositories are not compatible with (c)XSIBackup-DC.

#19 Re: General matters » XSI DC 1.4.3.17: job with --replica --config-backup half works » 2021-05-11 09:18:22

Maybe multipathing is the culprit, not an expert, but some round robin splitting system calls could be an issue. There should be some option for this in the iSCSI client configuration.

#20 Re: General matters » VM replication completes with segmentation fault » 2021-05-11 09:11:18

When some job ends abruptly, the remote .locked file in the root of the replica folder or repository is not deleted, just remove it manually to unlock.
--repair applies to repositories, has nothing to do with VMs.
If the segfault was generated after the backup process, then your backup is OK and you don't need to worry about its integrity.
Please, post the relevant code to find out.

#21 Re: General matters » Binary regexfilt missing from XSIBackup-Free 1.4.3.18? » 2021-05-11 09:06:57

Please post some output/ example, we can't figure out what you mean with enough precision to offer a detailed answer.

#22 Re: Feature requests & improvements » Queue an existing job » 2021-05-11 09:04:43

There are many turns we can take, we can't take them all at the same time though.

#23 Re: Feature requests & improvements » Add option to check and repair in background » 2021-05-11 09:02:02

Thank you for your feedback, we'll study adding a report at the end of --check & --repair options.
In regards to detaching, nothing prevents you from sending the job to the background:

# Send to the background, keep connected to the current 
# terminal though (the job would end if you close the TTY).
xsibackup --repair /path/to/repository &
# Send to the background & detach from current TTY
nohup xsibackup --repair /path/to/repository &

#25 Re: Feature requests & improvements » Log file readability » 2021-05-06 07:57:56

You are right, there's room for improvement in the log, we'll dedicate some time for it in the 1.5 branch.
Current log is just the output of the program, that's why it contains decoration, as it is thought to be monitored with tail -f, you have an independent properly formatted log for errors (error.log)

Board footer