©XSIBackup-Free: Free Backup Software for ©VMWare ©ESXi

Forum ©XSIBackup: ©VMWare ©ESXi Backup Software


You are not logged in.

#926 Re: © XSITools » IP Backup to Synology - error: SHA1 chunk hashes check failed » 2020-02-03 06:32:41

Read this first: [url=https://33hops.com/xsibackup-disk-checksum-verification-silent-corruption.html]Checksum verification and silent coruuption[/url]
Post your output, there's nothing we can say without knowing what you are trying to accomplish.

#927 Re: General matters » How to check back-up command state or success » 2020-01-30 11:11:17

If you disconnect your TTY you kill the job, unless you append the [b]&[/b] character at the end of it, which will in turn disconnect the running process from the TTY.

Output is redirected to STDOUT when you run a backup manually. If you want the output to be logged, just redirect it to the [b]xsibackup.log[/b] file:

./xsibackup --backup "VMs(RUNNING)" /vmfs/volumes/datastore2/backup-local >> /scratch/XSI/XSIBackup-DC/var/log/xsibackup.log 2>&1

If you have a working job in the command line, you can easily generate the job file by just appending --save-job=NNN to the job arguments

./xsibackup --backup "VMs(RUNNING)" /vmfs/volumes/datastore2/backup-local --use-smtp=1 --mail-to=me@myserver.com --save-job=001

Once you know your backup is working fine,add it to the [b]/scratch/XSI/XSIBackup-DC/var/spool/cron/root-crontab[/b] file, like:

5    2    *    *    *   /scratch/XSI/XSIBackup-DC/etc/jobs/001 > /dev/null 2>&1

And run [b]--update-cron[/b]

./xsibackup --update-cron

To have your schedule added to the ESXi crontab at [b]/var/spool/cron/crontabs/root[/b]

In linux you have tools like [b][url=https://www.gnu.org/software/screen/]Screen[/url][/b] that allow you to reconnect to a disconnected TTY, in ESXi there's no such thing, thus you have to redirect output to a file to achieve the same result.

You may simply watch a running command running by using [b]tail -f[/b] on the [b]xsibackup.log[/b] file, like:

tail -f /scratch/XSI/XSIBackup-DC/var/log/xsibackup.log

#928 Re: General matters » How do I use remote server backup function? » 2020-01-30 10:51:27

./xsibackup [action] [source] [target] [options]

1/ [b]--backup[/b] is the  [b][action][/b]
2/ [b]"VMs(Linux1, Windows2, OtherVM)"[/b] is the [b][source][/b]
3/ [b]root@my-server-to-backup:22:/vmfs/volumes/datastore1/my-vms[/b] is the [b][target][/b]
4/ [b]/vmfs/volumes/datastore1/my-vms[/b] is the remote path in your [b][target][/b] expression

Yes, there are lots of reasons: license management, logging, job management. You can remotely start any job by just invoking it through SSH, so you don't lose any feature.

We don't think so and we don't believe any survey would shed any light on this. Democracy can become a lethal weapon when it's misused.

#929 Re: General matters » trouble with XSIBackup/ESXi backups » 2020-01-29 14:51:45

Don't worry, just as long as you can workaround the inode overflow your old [b](c)XSITools[/b] repository will still be valid

#930 Re: General matters » trouble with XSIBackup/ESXi backups » 2020-01-29 13:40:12

I'm sorry, the short answer is: no, you can't reuse some Pro (c)XSItools repository. As stated it even uses a different compression algorithm.

#931 Re: General matters » trouble with XSIBackup/ESXi backups » 2020-01-29 13:35:03

We have commented that fact in every possible place, including the manual. In any case, even if we had not commented anything, the absence of information around some fact does not imply there is some default answer. Both programs have different names, use different command line syntax and different compression algorithms.

Old [b](c)XSITools[/b] repositories used a big block size (10-50MB). As subsequent backups were aligned with the previous ones, still some nice degree of deduplication compression was achieved. In case of [b](c)XSIBackup-DC[/b], the default block size is 1MB, thus some 150 GB of data would exceed the iNode limit in VMFS, which is around 130.000.

[b](c)VMWare[/b] does not clearly state whether [b]VMFS6[/b] allows a bigger number of iNodes. Some places it is somewhat revelaed a higher number may be present in [b]VMFS6[/b], but it's not clear.

We have indeed checked that limit and it seems to be in the order of millions, still VMFS-6 is very slow when compared to ext4 or XFS. This is not due to VMFS-6 not being good enough. It's due to the fact that it was designed to host a limited number of files: those comprising a virtual machine, and not to host millions of small files in a deduplicated repository.

That is why we recommend to use [b]ext4[/b] or [b]XFS[/b] file systems to hold [b](c)XSIBackup-DC[/b] repositories.

#932 Re: General matters » Help choosing correct product. » 2020-01-29 10:09:54

The trial will work for six hours after each reboot with VMs up to 60GB

#933 Re: General matters » trouble with XSIBackup/ESXi backups » 2020-01-29 10:08:17

Compression is turned on in the [b]etc/xsibackup.conf[/b] file by default, if you want compression enabled just don't use the [b]--compression[/b] argument.

The error [b]can't open lock file[/b] is letting you know you have some permission issue on your datastore. Even if you are using the [b]root[/b] user you may have some error in your NFS [b]exports[/b] file, should the datastore be NFS.

Could also be some trivial matter like a full disk, provided that: as per your feedback, the backup run for some time and then stopped with that message that's what it looks like.

#934 Re: General matters » xsibackup will not run - operation not permitted » 2020-01-28 19:59:31

[b]sh: ./xsibackup: Operation not permitted[/b] message is due to [b][url=https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.security.doc/GUID-5D5EE0D1-2596-43D7-95C8-0B29733191D9.html]secure boot[/url][/b] being turned to on, turn it off and reboot.

When it happens in ©ESXi 8 and above and you have already turned off secure boot, it can be due to execInstalledOnly policy being enabled in the host.

[url]https://33hops.com/xsibackup-backup-virtual-machines-in-esxi-8.html[/url]

©XSIBackup includes the logic to prevent the [b]operation not permitted[/b] message by deactivating the feature right before the backup and activating it right after the backup has taken place.

This is achieved by setting aliases in the ©ESXi host that invoke the neccessary commands to disable/ enable execInstalledOnly policy when running ©XSIBackup. For those aliases to be set up in your system you need to install ©XSIBackup using the installer.

#935 Re: General matters » XSIBackup 11.2.8 - where is jobs directory? » 2020-01-28 19:43:14

Create the jobs directory in the root folder and place your jobs there 000-999, i.e.:

/scratch/XSI/XSIBackup/jobs/001

If you run a job and it's working as you expect, i.e:

./xsibackup --backup-prog=Vmkfstools --backup-type=custom --backup-vms="VM1, VM2, VM3" --backup-point=/vmfs/volumes/backup1/my-backups

Just add [b]--save-job=NNN[/b] to the end of the command and it will be saved to [b]/scratch/XSI/XSIBackup/jobs/[/b], the jobs directory will be created should it not exist.

./xsibackup --backup-prog=Vmkfstools --backup-type=custom --backup-vms="VM1, VM2, VM3" --backup-point=/vmfs/volumes/backup1/my-backups --save-job=001

Then edit the [b]conf/root-crontab[/b] file, creating it if it doesn't exist, and place your schedules there, i.e.:

0 2 * * * /scratch/XSI/XSIBackup/jobs/001

The above to run your 001 job everyday at 2:00 in the night, then run

./xsibackup --install-cron

And then

./xsibackup --update-cron

To make your crontab permanent even after a reboot.

Once you have setup your job at [b]/scratch/XSI/XSIBackup/jobs/001[/b], some people add it directly to the [b]/var/spool/cron/crontabs/root[/b] file, which is the file holding the root cron schedules. The caveat in doing that is this file is not persistent across reboots, thus next time you reboot your ESXi server, the schedule will be lost and you will have to re-enter it.

Apart from that the [b]/scratch/XSI/XSIBackup/jobs/001[/b] file has 0600 permissions and in some ESXi versions you can't even edit it, so you have to delete it, create it again assign 0700 permissions, edit it and the change permissions back to 0600.

To avoid all that hassle, keep your cron schedules in the [b]conf/root-crontab[/b] file and run [b]./xsibackup --update-cron[/b] every time you modify it.

#936 Re: General matters » How do I use remote server backup function? » 2020-01-28 19:30:19

From the manual and the --help output:

./xsibackup [action] [source] [target] [options]

You need:

1 - Action, mainly: --backup or --replica
2 - Source: a local path or a set of VMs, i.e.: "VMs(Linux1, Windows2, OtherVM)"
3 - Target: a remote path

You are passing an action and a remote path as the source argument, which will never work. [b](c)XSIBackup-DC[/b] works at the host level, it backs up your VMs in a given ESXi host to a local or remote path. Thusly, in your case:

./xsibackup --backup "VMs(Linux1, Windows2, OtherVM)" root@my-server-to-backup:22:/vmfs/volumes/datastore1/my-vms 

The above command should do it, provided that you have installed [b]xsibackup[/b] on the other host and that its installed to /usr/bin, which might very well not be the case.

Again making the assumtion that your remote host is another ESXi host and that you have installed [b](c)XSIBackup-DC[/b] to the default recommended location on both sides, you could use:

/scratch/XSI/XSIBackup-DC/xsibackup --backup "VMs(Linux1, Windows2, OtherVM)" root@my-server-to-backup:22:/vmfs/volumes/datastore1/my-vms 

If you use the above command, latest version of XSIBackup (1.1.0.8) will look for the remote binary at /usr/bin, then it will try to locate it by running a which command and if the previous measures fail, it will try in the same absolute path you are passing in the client command, that is: [b]/scratch/XSI/XSIBackup-DC/xsibackup[/b], which is in turn the default recommended installation path.

Should all of the above fail, because the [b]xsibackup[/b] binary is not present at [b]/usr/bin[/b], it cannot be located by a [b]which[/b] command and the remote installation path is not the same than in the client machine, then just add the [b]--remote-path[/b] argument stating where the remote [b]xsibackup[/b] binary is, i.e.:

/scratch/XSI/XSIBackup-DC/xsibackup --backup "VMs(Linux1, Windows2, OtherVM)" root@my-server-to-backup:22:/vmfs/volumes/datastore1/my-vms --remote-path="/vmfs/volumes/datastore1/XSIBackup-DC/xsibackup"

The above asuming that you have installed to [b]/vmfs/volumes/datastore1/XSIBackup-DC[/b]

#937 Re: General matters » Help choosing correct product. » 2020-01-27 17:06:35

Undoubtedly [b](c)XSIBackup-DC[/b] will be your best option. From a conceptual point of view it is the same as [b](c)XSIBackup-Pro[/b], [b]DC[/b] is much faster though. It has been totally redesigned from scratch and reprogrammed in plain [b]C[/b] and [b]Assembler[/b]. Just try the Free version in some spare server, you won't use anything else since you do.

#938 Re: General matters » sh: ./xsibackup: Operation not permitted » 2020-01-27 16:00:34

Is your ESXi patched with latests release upgrades, is it a manufacturer version?. Always use official VMWare builds avoid manufacturer versions like the pest.

#939 Re: General matters » Where I can download actual version? » 2020-01-27 15:58:44

We are taking the time to add some more functionalities. It will be launched soon, a matter of days.

#940 Re: General matters » How do I use remote server backup function? » 2020-01-27 15:57:40

It is even simpler with [b](c)XSIBackup-DC[/b], it's just not exactly the same. There is a manual in the ./doc folder explaining how to do that.

Just parse a remote path in the form: [b]user@a.b.c.d:port:/path/somewhere[/b]

#941 Re: General matters » Error: Can't get VMId from name » 2020-01-27 15:54:13

Some VM has some characters that need to be escaped, as you are passing a meta VM definition (ALL), they can't be escaped.
Check your VM names and paths and eliminate bizarre characters such as (, ), !, ', ¨, etc...
Try to limit you names to letters, numbers hyphens and underscores. You may also use spaces and some other characters, but what's the point in doing such thing?

#942 Re: General matters » XSIBackup DC - multiple emails » 2020-01-23 09:08:04

We'll try to add this to some release in the next weeks.

#943 Re: General matters » xsibackup DC exit code » 2020-01-23 09:06:45

In regards to the naming issue. You don't need to use any tool, just check the files names, the .vmdk names the .vmx file is pointing to, the displayName property in the .vmx file too and check that the .vmdk files point to the right -flat.vmdk files. If you are using snapshots you will need to tweak the .vmsd file content as well. That's all there is to it, keep control with your own eye, or you will just add another layer of uncertainties to your setup.

UPDATE:
It seems that the exit code issue is fixed in v. 1.1.0.7, just visit the change log, we'll add this fix there before we launch 1.1.0.7 in a couple of days at most:
[url=https://33hops.com/xsibackup-datacenter-change-log.html](c)XSIBackup-DC Change Log[/url]

#944 Re: General matters » consolidating snapshots » 2020-01-23 08:58:44

Contact support and provide the full log corresponding to this backup job, this is not a general issue.

#945 (c)XSIBackup-Pro & Free Classic bug tracker » XSIBackup-DC: VMs with previous snapshots » 2020-01-21 12:27:30

admin
Replies: 0

When backing up or replicating VMs containing previous snapshots, no matter the number. [b](c)XSIBackup-DC[/b] will backup all snapshot files present in the VM folder. This will be done OK, so you are safe in regards to the integrity of the files. Nonetheless the latest .vmsd file will be saved, and this .vmsd file will contain information up to and including the last snapshot in the chain, which is the backup snapshot. Thus, the VM will many times run as it is, but you would need to manually fix the .vmsd file, deleting the last snapshot and decreasing the snapshot counter one unit to be able to consolidate and delete the snapshots in the target or restored VM.

We will fix this issue in version [b]1.1.0.7[/b]

#946 Re: General matters » xsibackup DC exit code » 2020-01-21 12:12:34

You are starting with a mistery, what was your previous problem?

In regards to the exit code. We still have to create a table of exit codes, they will most probably be associated with the current error codes. By now you can't rely on the exit code to integrate [b](c)XSIBackup-DC[/b] in wider scope scripts, you need to check e-mail reports or output.

#947 Re: General matters » consolidating snapshots » 2020-01-20 10:22:10

Given the fact that you are using a cold backup, all files should be copied as they are and your VM should start the same way it was doing in its primary location. Any previous snapshot file is left as it was before the backup job, [b](c)XSIBackup[/b] will not consolidate or do anything on them.

If you want to consolidate your snapshots before the backup, just delete them.

Check the .vmx file and arrange the disk paths to point to the right snapshot in case they are wrong.

[b](c)XSIBackup-Pro[/b] offers more options in regards to backup rotation, like [b][https://33hops.com/xsibackup-help-man-page.html#deldirs]--del-dirs[/url][/b]

#949 Re: General matters » Backup runs, but logfile empty » 2020-01-16 16:34:36

You have done nothing wrong. When you run

/vmfs/volumes/datastore1/xsi-dir/xsibackup --run-backup=001

The output is printed to screen, whereas when you run this

"/vmfs/volumes/datastore1/xsi-dir/xsibackup" \
--backup-point=/vmfs/volumes/Storage2TB/backup \
--backup-type=custom \
--backup-vms="Pamir_OES" \
--exec=yes >> "/vmfs/volumes/datastore1/xsi-dir/var/logs/xsibackup.log" 2>&1

The output is logged to [b]/vmfs/volumes/datastore1/xsi-dir/var/logs/xsibackup.log[/b]

#950 Re: General matters » Trial testing error » 2020-01-16 16:21:06

We try to offer as much detailed information as we can, a quick combined search in Google (xsibackup + some topic) will point you to the most relevant information. Let us know if you miss something important.

Deduplication breaks up data into little chunks. [b](c)XSIBackup-DC[/b], as opposed to other solutions which use key/value DBs, makes use of the file system to store data chunks. We find this to be much more convenient, in fact a FS is indeed a key/value DB in many ways. Not only that, but it's optimized for speed and reliability, thus adding a layer on top of it to perform the same task is somehow redundant.

Depending on how much data you accumulate into a deduplicated repository, you can end up hosting many millions of data chunks. Thus, the most important facts to take on account when employing block level deduplication ([b]--backup[/b] action) are: speed when searching a given block and the number of inodes, namely number of files and dirs you can host in a given FS.

VMFS is a FS which was designed to store a small number of huge files whereas other file systems were designed for a general purpose and some of them are optimized to host a big number of small files. Thus [b]VMFS[/b] has a limit of around 130.000 inodes, more than enough to host virtual disks, but very few to host deduplicated repositories.

This wasn't such a big concern when using [b](c)XSITools[/b] (--backup-prog=Xsitools:z) in [b](c)XSIBackup-Pro[/b], especially if using a block size of 50MB, as the 130.000 limit was still enough to host quite a few TB of real data. Nevertheless, when using a block size of 1MB (DC's default block size), this is indeed something to be concerned about.

Among the dozens of available FSs for NAS devices and Linux distros, [b]ext4[/b] and [b]XFS[/b] are the ones that yield the best performance in speed and at the same time they can host millions of files and folders, thus they are your best bets when it comes to host [b](c)XSIBackup-DC[/b]'s repositories.

Please, do note that you choose the tools you need to perform a given task based on their characteristics, not on their popularity or on their reputation. Many people like ZFS, and it can be a great choice when it comes to host some type of data in some given contexts, nonetheless it can become a terrible choice to host [b](c)XSITools[/b] repos.

Board footer