©XSIBackup-Free: Free Backup Software for ©VMWare ©ESXi

Forum ©XSIBackup: ©VMWare ©ESXi Backup Software


You are not logged in.

#351 Re: General matters » Replica XSIBackup-DC versions » 2021-08-01 00:10:47

The SIGTERM is most probably due to an empty variable, this is a side effect though.
This kind of error happens when the remote FS is not mounted properly or there's some bug in the NFS implementation, there have been some in the ESXi 6 branch regarding the use of NFS 4 from the shell.
Please switch to NFS 3 and try again. NFS 4.1 is indeed useless in this context.

#352 Re: General matters » Issue on ESXi 5.5: sh: syntax error: unterminated quoted string » 2021-08-01 00:06:26

It is clear that you can't execute shell commands in the remote system. Make sure that you haven't activated secure boot and that the shell is working, you may need to reinstall ESXi.

There have been shell bugs in the past, try to upgrade your remote ESXi system to the latest patch in the branch.

#353 Re: General matters » Issue on ESXi 5.5: sh: syntax error: unterminated quoted string » 2021-07-31 09:56:54

What is this?: "> 1"
Why is the --mail-to argument missing the first "--"?

You are most probably generating your own errors by using a malformed job syntax.
In fact we believe it's quite remarkable that the job took place in the end.

Please, run this and post the output

./xsibackup --replica=cbt "VMs(Lecturas)" root@REDACTED:22:/vmfs/volumes/Backups/backups_ctec

#354 Re: Feature requests & improvements » Add some email content flexibility » 2021-07-30 14:57:33

As said we will improve the flexibility of the e-mail report, still, you should not rely on e-mail to perform that kind of logging.

You have the [b]var/log/backupdb.log[/b] file where all those variables are logged into a single line, it's easy to take them from there using bash or Python and put them anywhere: Syslog, DB, etc...

#355 Re: General matters » XSIBackup-DC: Is restoring differentially supported? » 2021-07-30 14:50:34

The best way to save in offsite storage is to upload the deduplicated repository using Rsync. Turn the Rsync flags to avoid a full cheksum and only compare by size and date and you will have an extremely fast sync to the offsite server while keeping a compression ratio close to 99%, thus 1TB could easily host some 90TB of real data.

We don't know what type of OS you are using as the offsite backend, a Linux distro would be the perfect match though, we still recommend CentOS/ RHEL 7

XSIFS performance depends on the resources you have. You should be able to make it work over NFS with a more than decent throughput for big files: 20-30MB/s, that's what we get on commodity hardware. Using SSD would drastically improve that.

#356 Re: General matters » Issue on ESXi 5.5: sh: syntax error: unterminated quoted string » 2021-07-30 14:30:45

What is the job you are executing?.

You have some kind of bash error. We have tested the software today from ESXi 5.5 to 6.5 with no issue.
Prepend strace to your job to get some detailed system call level output.

strace ./xsibackup ...

#357 Re: General matters » XSIBackup-DC: Is restoring differentially supported? » 2021-07-30 14:19:47

No, there isn't.
Use a replica and upload the replica or upload the repository instead of the rebuilt VMs.

#358 Re: General matters » Issue on ESXi 5.5: sh: syntax error: unterminated quoted string » 2021-07-30 07:23:02

We aren't quite getting you.
Why do you expect the CBT backup to work when you eliminate the .ctk files?
Why do you expect a --replica to continue to work when you rename files manually?

It's like removing the wheels of your car and then complaining because it doesn't move.
Asuming the rename was done right (that's a different story) you must completely reinitialize the replica.
To reset the CTK files you have the --reset-cbt argument.

#359 (c)XSIBackup-Pro & DC bug tracker » Bug in --save-job argument » 2021-07-29 17:00:47

admin
Replies: 0

The --save-job argument doubles the xsibackup binary last path token, like in the example below.

/vmfs/volumes/5eb05020-68ce0321-a71a-a0369f57b9e0/XSI/XSIBackup-DC/xsibackup/xsibackup \
--backup \
"VMs(WDES01)" \
"root@192.168.3.34:22:/volume4/backup4TB/DATACENTER-BACKUP-1M-F" \
>> /vmfs/volumes/5eb05020-68ce0321-a71a-a0369f57b9e0/XSI/XSIBackup-DC/var/log/xsibackup.log 2>&1

We have already fixed this, it will be available in 1.5.1.0

#360 Re: General matters » Jobs are saved with wrong executable path » 2021-07-29 16:25:36

OK, thank you for the feedback, we'll check the --save-job argument.

UPDATE:
We already fixed it. It will be available in next release.

#362 Re: General matters » XSIBackup-DC is it possible to --check on a remote over SSH replica? » 2021-07-28 16:58:41

Not really, just a sequence integer associated to each block or zone in the disk. Given that you have the base data already on the remote end, transfering blocks greater than some value ensures that the remote disks contain the same changed data.

If some, let's say, ransomware infects your source disk you lose everything.

The only way to keep an unlimited number of real restore points is to use the --backup argument. Then using xsifs you can mount any of the restore points in Linux and access the data contained within them.

There's already a Beta preview of 1.5.1.0 featuring CBT on the --backup as well as in the --replica action.

#363 Re: General matters » XSIBackup-DC is it possible to --check on a remote over SSH replica? » 2021-07-28 15:57:17

Just use some sequence number above 1, that should do it. If you had performed many CBT syncs, the closer to the failing sequence, the lower the amount of data to transfer. It might not be significant in your case, we don't know the amount of data generated in between CBT cycles.

SSH compression is disabled by default in both --replica and --backup, it's redundant and useless. In case of --backup because the data chunks themselves are already compressed by default. In case of --replica because zero blocks are filtered out and compression is only really significant when many zeros are present.

Compressing binary data where most zeros have already been stripped off makes no sense.

#364 Re: General matters » XSIBackup-DC is it possible to --check on a remote over SSH replica? » 2021-07-28 15:25:06

Then you should resync the whole VM. Delete all files at the target.
If you want some faster kind of replica and finer control on the data to transfer, you can use CBT (DC version only).
The first replica copies all data, then the CBT feature copies over just the blocks that have changed and skips the rest.
If later on you have some error of this kind, you can just use a lower CBT sequence number to resend all blocks from a given point in time.
[url=https://33hops.com/xsibackup-cbt-implementation.html]Details on (c)XSIBackup CBT implementation[/url]

#365 Re: General matters » XSIBackup-DC is it possible to --check on a remote over SSH replica? » 2021-07-28 12:38:06

Great!, they block basic system calls. This must be the ultimate in security. Wait until they discover that blocking all of them is even more secure.

Our recommended distro is CentOS. Now that they took the turn of eliminating the stable CentOS releases we will switch to some other stable distro, most probably based in Debian.

We meant that you share the output of the --replica job that failed.

--check argument just checks that the blocks in the manifest exist in the replica and their hashes are correct. If you use the --replica=cbt plus --options=R, your remote VM will be registered and you can switch it on to check its contents.

If you receive some error in the middle of a backup, you should investigate where that error came from and whether that corrupted your mirror disks. As you seem to be quite confident, we guesss you received the error after 100% of the blocks had already been copied.

#366 Re: General matters » Jobs are saved with wrong executable path » 2021-07-28 11:18:20

You are refering to the GUI, right?
We haven't detected anyhting similar so far. We'll take a closer look, maybe a combination of actions could result in that spurious string.

#367 Re: General matters » XSIBackup-DC is it possible to --check on a remote over SSH replica? » 2021-07-28 10:58:57

Can you share the output?, otherwise we can just especulate.
--check command is local, run from the remote Linux server.

#368 Re: General matters » Issue on ESXi 5.5: sh: syntax error: unterminated quoted string » 2021-07-28 10:30:17

Did '/vmfs/volumes/Backups/backups_ctec' exist in the remote FS?
It all seems related to an inexistent remote dir, although it should have been created previously.

You may encounter problems when trying to create an SSH tunnel between too distant versions of OpenSSH, as most if not all KEX algorithms and ciphers may have been deprecated.

In this case it looks like they were able to agree on some minimum common denominator in terms of KEX & ciphers, as the remote binary could be updated.

#369 Re: Feature requests & improvements » Add some email content flexibility » 2021-07-28 10:19:31

We will add new features in this direction in future releases. You can nonetheless design your own e-mail templates.

[url]https://33hops.com/xsibackup-dc-full-manual-home.html#email-reports[/url]

#370 Re: General matters » Replica XSIBackup-DC versions » 2021-07-24 13:13:21

We totally agree. (c)XSIBackup is a tool, you can use it from a basic approach to a much complex one. We offer a man page and post use cases that we find to be interesting as often as we can. We manufacture a swiss knife, it's up to you how to use it though.

We aren't by now pretending to offer anything beyond that.

#371 Re: General matters » Replica XSIBackup-DC versions » 2021-07-24 10:04:31

You are right, taking the user's hand and walking him through all the learning curve is the way to go.

Still, given our current size we can't afford to do so, thus, by now we will have to treat our users as Linux sys admins with a mid knowledge of the Linux command line environment.

Once we reach our goals we can move up the commercial line towards the end user, or at least power users. Take on account that given the nature of the software's purpose, we may never reach end users, it might even be couterproductive.

#372 Re: General matters » Issue on ESXi 5.5: sh: syntax error: unterminated quoted string » 2021-07-24 09:57:32

Thank you for your feedback.

Always use the latest release in a branch, in your case ESXi550-201809001 or [b]ESXi U3k[/b].
There have been multiple issues related to shell commands in the past.
Just avoid rotating by size until this is fixed.

We'll revise this to see if we can do something to work the problem around.

#373 Re: General matters » Replica XSIBackup-DC versions » 2021-07-23 12:40:43

It is indeed working for us

 ./xsibackup --replica "VMs(WS2012)" "root@192.168.133.23:22:/volume1/backup/replicas/$(date +%Y%m%d%H%M%S)" --rotate=3

|---------------------------------------------------------------------------------|
||-------------------------------------------------------------------------------||
|||   (c)XSIBackup-DC 1.5.0.11: Backup & Replication Software                   |||
|||   (c)33HOPS, Sistemas de Informacion y Redes, S.L. | All Rights Reserved    |||
||-------------------------------------------------------------------------------||
|---------------------------------------------------------------------------------|
                   (c)Daniel J. Garcia Fidalgo | info@33hops.com
|---------------------------------------------------------------------------------|
System Information: ESXi, Kernel 6 Major 5 Minor 0 Patch 0
-------------------------------------------------------------------------------------------------------------

...
...
...
...
...
...
...
Total size:                                                                    100.02 GB    | Done 100.00%
-------------------------------------------------------------------------------------------------------------
*** Snapshot was removed ***
-------------------------------------------------------------------------------------------------------------
Backup end date: 2021-07-23T12:38:10
-------------------------------------------------------------------------------------------------------------
Time taken: 00:06:58 (418 sec.)
-------------------------------------------------------------------------------------------------------------
Total time:      418 sec.
-------------------------------------------------------------------------------------------------------------
Full file speed:                                                                           245.01 mb/s
-------------------------------------------------------------------------------------------------------------
Real data speed:                                                                            33.42 mb/s
-------------------------------------------------------------------------------------------------------------
Item backup completed without errors
-------------------------------------------------------------------------------------------------------------
Final checksum: 10095909563 bytes were sent and confirmed to have been written remotely
-------------------------------------------------------------------------------------------------------------
Data processing completed successfully
-------------------------------------------------------------------------------------------------------------
--rotate option was detected, retrieving backups to prune...
-------------------------------------------------------------------------------------------------------------
- Deleting replica at '/volume1/backup/replicas/20210723091155'
-------------------------------------------------------------------------------------------------------------
Removed host <tmp> dir        OK
-------------------------------------------------------------------------------------------------------------
Removed prog <tmp> dir        OK
-------------------------------------------------------------------------------------------------------------
Unlocked backup               OK
-------------------------------------------------------------------------------------------------------------

#374 Re: General matters » Replica XSIBackup-DC versions » 2021-07-22 21:03:34

You can now since version 1.5.0.11. Previous version didn't support rotating sets of replicas.

#375 Re: General matters » Replica XSIBackup-DC versions » 2021-07-22 20:25:54

We have indeed embraced that type of usage as explained in the above post.

[url=https://33hops.com/xsibackup-extended-rotation-features.html]Extended rotation features[/url]

I believe that the key matter here is that you probably have not perceived the radical difference between a simple timestamp and a Julian + Mod rotation.

When you use [b]$(( $(date +%j) % N ))[/b], the rotation is implicit to the date algorithm. You don't need to use argument --rotate, the date will always produce a fixed rotation from 0 to N-1, thus it will revisit each folder each (N-1) times and produce a differential copy. The first time for each of the integers produced the copy will be full.

When you use the plain time stamp pattern, each folder is new every time and so your VM is fully copied every time, there isn't any kind of differential feature.

Of course it's not possible to generate the data out of thin air. If there isn't any seed data in the target folder, no differential copy can be performed.

Board footer