©XSIBackup-Free: Free Backup Software for ©VMWare ©ESXi

Forum ©XSIBackup: ©VMWare ©ESXi Backup Software


You are not logged in.

#501 Re: General matters » XSIBackup-DC --replica=cbt: Error code 194 at file signal.c, line 194 » 2021-05-17 09:23:21

Again, you are not providing the full command you have used.
Check your (c)ESXi host for errors, in special when creating the snapshot.

#502 Re: General matters » XSIBackup-DC --replica=cbt: --options=R appears to be without function » 2021-05-17 09:21:39

Please, provide both the full command and its associated output every time.
Just make sure that you have written a well formed job command any run the CBT replica again, you are just reporting a first full copy.

#503 Re: General matters » XSIBackup-DC: Permission denied at Synology NAS after --add-key » 2021-05-16 19:47:44

You didn't say whether you were able to exchange keys correctly, nor what was causing your previous issue.
You aren't providing the jobs that produced the output you are posting, which makes almost impossible to offer some clue.
You don't need to pass the --remote-path argument, /bin and /usr/bin (which is usually just a hardlink to the former) are sought by default.

#504 Re: General matters » XSIBackup-DC: SMTP error 550 when using option --subject » 2021-05-16 18:17:37

You can run the program with option --verbosity=10, nonetheless the finest level of detail you will find about that error is:

[b]550 5.0.0 SUBJECTBLOCK This mail is a virus or spam or a mail loop[/b].

This is coming from the SMTP protocol conversation and is definitive.

#505 Re: General matters » XSIBackup-DC: tmp directories not removed after backup » 2021-05-16 18:15:56

Client/server will always be faster, as the CPU load is balanced between client and server. That will depend on if you are able to configure key authentication, which seems to be a problem with your DSM version default configuration.

#506 Re: General matters » XSIBackup-DC: Permission denied at Synology NAS after --add-key » 2021-05-16 18:12:21

You will always need to enter your password, unless you tell the ssh client to use the exchanged key by using the -i option.

ssh -i xsibackup_id_rsa root@nas

Add the key and run the above code from the (c)XSIBackup-DC installation root. If you are not asked for a password, then your key exchange worked.

If the automatic login does not work either, run the command with the -vvv option to get the detail on the OpenSSH protocol conversation and the detail on why key authentication isn't working.

ssh -vvv -i xsibackup_id_rsa root@nas

When you run --add-key again after a successful key exchange, you should see a confirmation that the key is in the remote system and that key authentication is enabled.

You don't say what version of DSM you are using. Being able to use Synology as a backup device over IP depends on the DSM desing and SSH configuration options. Double check whether [b]/root/.ssh/authorized_keys[/b] is the right place to store your public keys and check whether key authentication is enabled in the [b]/etc/ssh/sshd_config[/b] file.

#507 Re: General matters » XSIBackup-DC: SMTP error 550 when using option --subject » 2021-05-16 18:02:33

Start by changing the --subject text to see if you receive the same response.

#508 Re: General matters » XSIBackup-DC: tmp directories not removed after backup » 2021-05-16 18:00:37

We haven't tested (c)XSIBackup-DC on BTRFS, it's still under development, you should not use it in production, it will not be the best approach, in special if you have turned on deduplication.

(c)XSIBackup-DC has its own buil-in deduplication engine, you don't need additional deduplication. I believe many users don't realize what deduplicating data in real time with a small block size as BTRFS does means in terms of CPU and memory. If on top of that you add that Synology doesn't ussually have specially powerful CPUs nor much RAM..., you are creating the perfect storm.

Use XFS or ext4, as stated all over (c)XSIBackup-DC posts and manual.

(c)XSIBackup-DC has been designed to run in the (c)ESXi host. Running it in the Synology device is a terrible idea, please, don't do that.

#509 Re: General matters » XSIBackup-DC: run --replica and make --backup from replicated VM » 2021-05-15 10:33:50

There isn't any drawback, in fact it's our recommended procedure to backup big production VMs.

(c)XSIBackup 1.5.0.0 and above offers [b][url=https://33hops.com/xsibackup-cbt-implementation.html]CBT compatibility[/url][/b] for replicas, thus you can use the same approach that you used with Onediff, yet with a more advanced algorithm, as Onediff depended on a permanent backup snapshot on the source VM.

As with Onediff, you depend on the first part of the scheme working right.

[b][url=https://33hops.com/xsibackup-dc-full-manual-home.html#cbt]CBT feature[/url][/b] offers the option [b]--options=R[/b], which will create a test snapshot after each CBT round and will allow you to turn on the replicated VM without affecting the CBT replica.

./xsibackup --replica=cbt "VMs(WS2012)" root@192.168.3.83:22:/vmfs/volumes/datastore1 --options=R

#510 Re: General matters » XSIBackup-DC: tmp directories not removed after backup » 2021-05-15 10:26:57

We can always improve the source to try to handle unexpected situations better, that said: many of your issues point at some kind of sluggish behavior from part of your FS. We have had all kind of situations with users, xsibackup will not prevent you from doing something awkward, like backing up to some deduplicated FUSE FS, nonetheless that will not yield anything close to good results, as you would be producing such a load on the target FS that the mere fact that you could end some backup would be a miracle, yet not achieving any productive result.

The same happens when performing backups to VMFS6, we won't even comment on VMFS5, as it can handle just around 130,000 inodes, which is clearly insufficient to host any deduplicated repository.

VMFS6 can host millions of files (the number of inodes is not official, we have benchmarked it though), still VMFS6 has not been conceived to host millions of small files, but a few huge ones, which is right the opposite. Thus, you can't expect a good speed from it, in fact it's slow as hell when compared to some regular FS such as XFS or ext4. So, if you try to force VMFS6 to do something beyond its design principles, you might take it to its limits and start producing awkward results without even noticing you are doing something wrong.

We always take the time to explain this things in most posts covering backup targets.

#511 Re: General matters » XSI DC 1.4.3.17: job with --replica --config-backup half works » 2021-05-15 10:15:28

If you can steadily reproduce your error, you should first prepend strace to your job, run it and post at least the end of the output.

strace ./xsibackup --backup ....

#512 Re: General matters » XSIBackup-DC: No email report sent when xsibackup detects an error » 2021-05-15 10:08:16

Don't use the --backup-how option if you want to perform hot backups, it's the default behavior.
11 is the received signal number, that is: a SEGFAULT. Which is caused when trying to access a restricted area of memory. It seems that due to the failed snapshot that condition is generated in your case. We should delve into it to at least handle the error in a nicer way and show some hint along with the error.

We have two general design principles considerations that are colliding and entering a contradiction there, which are: always trying to continue, no matter what happens to, at least, copy as much data as possible, with: control errors and show nice and explanatory responses. We should probably just continue to the next VM in case of some errors like a failed snapshot, as trying to continue with the backup process will just lead to undefined behavior.

The e-mail report is not sent due to the program jumping to the uncontrolled error handling function, namely: the program exists abruptly. We will add e-mail reporting to it in future versions.

#513 Re: General matters » XSI DC 1.4.3.17: job with --replica --config-backup half works » 2021-05-15 09:52:08

As stated, we are talking about a simple atomic mkdir system call, not any complicated set of commands. Strace will offer you all the details on the issue. It could be that some DS in async mode would not create the folder in time for the subsequent commands to be able to copy to it. If that was the case, switching to Sync mode would solve the issue.

We could add a helper to wait until we detect the folder, but that would be misleading, as the rest of the program works in Sync mode. Could be some other thing, we'll keep investigating, we haven't been able to reproduce the issue though.

#514 Re: General matters » XSI DC 1.4.3.17: job with --replica --config-backup half works » 2021-05-14 18:18:41

This issue is not something obscure. It's a simple system call (mkdir) that creates that dir. You can run some job with strace (just prepend 'strace' to it) to detect the exact reason why the system call fails. It will produce a lot of output though, and will be slower.

Are you using some kind of Async option maybe?

#515 Re: General matters » VM replication completes with segmentation fault » 2021-05-14 18:08:35

It's not something about the data files, but some variable which is expected to contain some value but does not or is overflown.
It could be something as simple as some string length being overflown, some unexpected value in some field of any of the VM config files, etc..
This is not a generalized issue, thus, there must be something in your job/ environment triggering that behavior.

Provide as much feedback as you can. Start by posting all relevant information, that consists of the following:

1 - The full job that caused the issue.
2 - The full output
3 - Any relevant information, such as some unusual length in some path, email string, etc...

#516 Re: General matters » Binary regexfilt missing from XSIBackup-Free 1.4.3.18? » 2021-05-14 18:02:27

(c)XSIBackup-DC 1.5.0.3 has been published and the REGEXP simulator feature of the --exclude argument has been fixed.

#517 Re: General matters » Binary regexfilt missing from XSIBackup-Free 1.4.3.18? » 2021-05-14 17:36:12

There's still an issue with that REGEXP exclusion part of the GUI. It does not work.
It will be fixed in 1.5.0.3 later today.

It is the helper to simulate the disks/files that will be excluded that fails, not the exclude mechanism itself.
You can easily add that in the job file and test it in the command line.

#518 Re: © OneDiff » Impact on backup chain if sometimes the hash comparison fails » 2021-05-14 15:10:16

I was still writing when you answered, don't miss the rest of it.

Yes, that assumption is correct.

Most, if not all, software manufacturers don't offer a checksum comparison mechanism on such big files as virtual disks. The reason is that most people won't try to understand how things work, they will just call their support help desks and drive them crazy, assuming that a wrong checksum means the backup failed. Thus, it's probably not feasible to offer such characteristic to the great public, the same way that it would be pointless to try to explain people that things are literally 'not there' when nobody looks at them.

#519 Re: © OneDiff » Impact on backup chain if sometimes the hash comparison fails » 2021-05-14 14:56:13

First of all you must know that (c)XSIBackup-Pro Classic has been deprecated, we only sell editions based in DC technology which is more advanced from all points of view.

You are making some wrong assumptions which in the end turn out to be half right and some right ones.

[b]Wrong:[/b]

Onediff does not produce a full backup every time. It coalesces the differential data stored in an intermediate snapshot with the seed data available from previous backup cycles. Thus, if for some reason that differential snapshot fails at the time to be integrated with the previous seed data, you will end up with a wrong backup -flat.vmdk file.

To read: [https://33hops.com/xsibackup-disk-checksum-verification-silent-corruption.html]A post on silent corruption with extended information[/url]

In the end your assumption turns out to be half right, as most of the checksum errors are due to read errors, that would mean that the base data is indeed O.K.. On top of that they tend to be sticky, thus you may repeat a failed checksum and get the same result, which is double puzzling.

The proof of that is that you may receive a wrong checksum in one backup cycle, but still get a good one on some subsequent one. As the checksum is indeed done on the full length of the -flat.vmdk file, a valid checksum comparison means that your data is indeed the same on both sides. That could only be possible if the previous failed checksums were spurious.


[b]Right:[/b]

The possibility that you have a coincidence in a hash by chance is despicable, thus we can assume that any block with a given hash represents the data there contained, with the exception of a silent write or compression error, which is also quite despicable, as there are a multitude of intermediate checks which would need to all fail for that to happen.

Per instance, let's say that you read a chunk of data get its checksum, compress it and send it over the wire. If you get no error, that means that the data was read without an error via a system call with its own integrity mechanisms, then a checksum was calculated for it, also with its own integrity mechanisms, then the chunk is sent over an SSH tunnel with integrity checks and the tunneled data is in turn eventually sent over TCP to a temp file which is renamed at the end of the transmission only if no error was returned.

You can assume your stored blocks are as OK as your disks are.

Thus, if you receive an error in some XSITools deduplicated backup and it is interrupted. The previous and subsequent backup cycles which do not return any errors will be OK, as will be compounded by error free blocks.

Nonetheless you may end up with some garbage, if some block in a broken backup is not used any more by any deduplicated set.

#520 Re: General matters » XSIBackup-DC --replica: Keep the latest n versions of replicated VMs » 2021-05-14 14:41:13

You are probably assuming that DC works exactly the same way as Pro Classic, which is incorrect.
DC works by two main actions: --replica & --backup.

Replica produces replicas, it's not been thought to produce full backups, as replicas (and also backups) are differential and  if you create a new replica every time, you won't be taking advantage of any delta algorithm.

In any case if you still wish to do so, it's very easy, just add some dynamic replica target such as:

./xsibackup --replica "VMs(MyVM)" /vmfs/volumes/backups/replicas/$(date +%Y%m%d)

#521 Re: General matters » XSIBackup-DC: SMTP error 550 when using option --subject » 2021-05-14 14:35:17

You are receiving that error from the SMTP server, you probably added some text that triggered that response.

#522 Re: General matters » XSIBackup-DC --backup: Use of --rotate in a deduplicated backup repo » 2021-05-14 14:33:18

--rotate in a deduplicated repository is equivalent to pruning as much backups as to satisfy the rotation figure. Use to your discretion.

#523 Re: General matters » XSIBackup-DC: dir cfgbak only created with --backup and not --replica » 2021-05-14 14:31:27

We'll revise that behavior, by now, just create it manually before the replica.

#524 Re: General matters » How to keep multiple VMs powered off until cold backup is finished » 2021-05-14 14:28:38

You would need to script that, e.g.: turn off all VMs to backup, then turn them on when you are done.

#525 Re: General matters » Binary regexfilt missing from XSIBackup-Free 1.4.3.18? » 2021-05-14 14:27:19

Yes, that was probably a mistake at the time to pack latest versions. We have re-added the binary regexfilt to 1.5.0.2, just published.

Board footer