#1 Re: General matters » Cron problem » 2019-01-23 18:05:38

Many people, including ourselves, are using this cron. It is indeed working for you, as the test file demonstrates. It should not be that difficult to debug where your problem is:

1 - Backup your current crontab
2 - Call your crontab manually: # /var/spool/cron/crontabs/root
3 - Tail -f your xsibackup.log file in a different SSH window to see if something is being written there:
tail -f /vmfs/volumes/datastore1/xsi-dir/var/logs/xsibackup.log
4 - Reset permissions on your crontab:
chmod 0600 /var/spool/cron/crontabs/root
5 - Set some new execution time in the crontab.
6 - Check your tail -f window.
7 - If it still doesn't work open a new SSH window and tail -f your ESXi server's logs:
tail -f /scratch/log/syslog.log
...

And don't forget to check if there are ongoing XSIBackup processes by using ps -c, so that you don't pile processes up and clog execution.
ps -c | grep xsi

#3 Re: General matters » Guidance for Backup Process » 2018-03-16 20:49:35

If you just want an incremental backup, as fast as possible, use (c)OneDiff. You can optionally use --certify-backup=yes to get a hash certification proof upon backup completion.


/path/to/xsibackup --backupid=00 --time="Mon 00:00|Tue 00:00|Wed 00:00|Thu 00:00|Fri 00:00|Sat 00:00|Sun 00:00|Mon 12:00|Tue 12:00|Wed 12:00|Thu 12:00|Fri 12:00|Sat 12:00" --backup-prog=onediff --backup-point="/vmfs/volumes/backup1" --backup-type=custom --backup-vms="W701,Linux01" --mail-to=you@yourdomain.com --use-smtp=1 --certify-backup=yes

If you want an incremental backup but also archiving your VMs with deduplication and a high space utilization level use (c)XSITools program with compression enabled. You can also optionally use --certify-backup=yes


/path/to/xsibackup --backupid=01 --time="Mon 00:00|Tue 00:00|Wed 00:00|Thu 00:00|Fri 00:00|Sat 00:00|Sun 00:00|Mon 12:00|Tue 12:00|Wed 12:00|Thu 12:00|Fri 12:00|Sat 12:00" --backup-prog=xsitools:z --backup-point="/vmfs/volumes/backup1" --backup-type=custom --backup-vms="W701,Linux01" --mail-to=you@yourdomain.com --use-smtp=1 --certify-backup=yes

As you want an additional full backup on Sundays, we'll add Sunday's noon backup to a different job that will use --on-success and --on-error event handlers to trigger another backup job upon finishing. This full backup job will not need the --time argument, as will be triggered by a previous backup job. --certify-backup=yes is optional in all cases.


/path/to/xsibackup --backupid=02 --time="Sun 12:00" --backup-prog=xsitools:z --backup-point="/vmfs/volumes/backup1" --backup-type=custom --backup-vms="W701,Linux01" --mail-to=you@yourdomain.com --use-smtp=1 --certify-backup=yes --on-success="backupid->03" --on-error="backupid->03"


/path/to/xsibackup --backupid=03 --backup-prog=vmkfstools --date-dir=yes --backup-point="/vmfs/volumes/backup2" --backup-type=custom --backup-vms="W701,Linux01" --mail-to=you@yourdomain.com --use-smtp=1 --certify-backup=yes

#4 Re: General matters » Licenses » 2018-02-28 20:09:06

No, just as long as you use one single crontab. Install XSIBACKUP-PRO to any of those two servers and use --host option to launch backup jobs on the other one.

#5 Re: General matters » Hot backup: Is consolidate on destination posisble? » 2018-01-21 22:09:03

You can backup to a datastore or you can create a script that deletes your old snapshots after the backup cycle has finished.

#6 Re: General matters » xsibackup and plink.exe » 2018-01-21 22:02:46

What do you mean by "nothing is happening". Have you turned on debuging?, how are you parsing the password, manually?
Come on!, "everything else is working except XSIBackup", what is everything else?. XSIBackup is a bash script, so it will behave as any other bash script, there's nothing that prevents executing it remotely, just turn on debuging and narrow the cause down.

#7 Re: General matters » Confused on remote backup » 2018-01-19 07:36:40

I guess you mean that the remote baackup halts there.
There's a small bug that could affect remote backups, we just sent you a fix.
In any case, it could be that XSIBackup is not able ro determine the remote FS type. Contact support if you still have the problem.

#8 Re: General matters » Log Rotate » 2017-11-24 19:42:08

Are you sure?, 271 gb. is a looooot of text, you must be backing up many VMs at a very high frequency rate. Even in that case, still looks a lot to me.

We'll add some log rotate feature in some future version, but, you just have to rename the elder and XSIBackup will create a new one.

#9 Re: General matters » AWS SES issue w/ authentication » 2017-11-21 14:59:32

That's way below the error:

535 Authentication Credentials Invalid

It makes no sense to try to interpret a derived error message before actually fixing the primary one.

#10 Re: © OneDiff » error OVMXCHK1: VMX file check » 2017-10-15 18:16:52

We really don't know how to help you with this, nobody else seems to have this problem. On the other side, the matter is quite obvious.

#11 Re: General matters » REGEXP for VM selection doesnt appear to work. » 2017-10-13 10:46:20

The regexp feature will come out with v 10.0.4, still preparing.

#12 Re: General matters » Permission Denied error » 2017-09-12 17:33:33

You don't execute programs in the command line this way

/vmfs/volumes/datastore1/xsi-dir/ ./xsibackup

Id, I don't want you to feel you are not getting answers, but this whole topic makes no sense. This is basic Linux command line knowledge.

1 - You can execute a program calling it by absolute path.

/vmfs/volumes/datastore1/xsi-dir/xsibackup

2 - Or from its containing folder.

./xsibackup

The way you are trying to execute the program is plainly wrong.

#13 Re: General matters » Unable to send the backup report via email » 2017-09-11 14:17:19

You need to configure your e-mail server to allow pipelining, you probably have some parameter like reject_unauth_pipelining activated somewhere. Use this post as a starting point https://groups.google.com/forum/#!topic … Sd4mp0katY

In any case, if you are not an expert mailmaster, just use any gmail.com, hotmail.com, yahoo.com, etc... account, we have tested this servers with XSIBackup and they are free to use.

#14 Re: General matters » Need some help deciding if XSI backup is right for my needs » 2017-09-11 14:08:10

Follow this tutorial: https://33hops.com/xsibackup-pro-onedif … olicy.html
If you don't want deduplication, just use Rsync to backup the OneDiff mirrored _XSIBAK.
In any case, why not using deduplication if you have it available and on top of that you already have a fresh backup?

#15 Re: General matters » hotbackup of vm on esxi free » 2017-09-11 09:21:13

Yes, of course, that is basic XSIBackup functioning. XSIBackup will not even notice the difference between the Free or licensed version of ESXi.

#16 Re: Rsync » Setting the remote user on a rsync backup ? » 2017-08-20 15:23:35

You are assuming XSIBackup uses the rsync:// protocol directly, when it does not.

XSIBackup tunnels Rsync protocol through an SSH tunnel, thus, you must make sure a compatible Rsync binary is present at the other end's path (/bin or /usr/bin) and that you have previously linked both servers via the --link command

Close versions of Rsync are generally compatible, using the same version on both ends is obviously the best thing to do.

#17 Re: General matters » Pre sales question - Can XSI do this for me and how » 2017-08-02 14:02:37

XSIBackup free will allow you to backup to both, local and remote dedicated servers. XSIBACKUP-PRO will allow to transfer just changed blocks to the remote cloud server without any delta checksum lag. Upcoming XSIBACKUP-PRO 10.0.0 will combine those characteristics with XSIDiff which will make first upload much faster than Rsync by just copying used blocks.

VPSs are a cheap option to host some HTTP or e-mail server, definetely not something to use as VM cloud backup storage. CentOS 7 is not as production proven as CentOS 6.8 and ESXi 6.5.0 is not the best production ready ESXi distribution available. Choosing the right versions of an OS to use in production requires a lot of testing and benchmarking. Happily, you have a lot of reviews and forums available in the Internet where you can retrieve information about what are the best choices available.

As a rule of thumb: never use the latest version of anything, unless it has empirically proven to be stable enough and offers some new feature that you objectively need.

XSIBackup allows you to direcly use a VM backup, it's ready to be switched on in the backup location, unless you use Borg or XSITools as the backup end, which will require you to restore the backup by using the restore module. It's an easy operation, just running a command will restore your VM wherever you want at LAN speeds +- 60 mb/s.

If you backup in a cloud, make some quick figures to have an aproximate idea of how long would it take to restore your backups. Per instance, a 100 gb. VM would take around 6 h. to complete, provided that you have a 100 mbps FO that can sustain an average of 4 to 5 mb/s

#18 Re: General matters » Cron » 2017-08-02 13:45:05

Your ESXi installation might be broken.

https://sourceforge.net/p/xsibackup/dis … /b16e5499/

Reboot and if the problem persists then reinstall.

#19 Re: General matters » Restart after CRON install » 2017-07-31 14:48:36

There's always a first time for everything ;-)

#20 Re: General matters » sometimes KO, Failed to clone disk: The file specified is not a virtu » 2017-07-31 07:33:21

Check that no other program is accessing the file during the vmkfstools cloning process.

ps -c | grep "your-file"

Ommit the extension so that both .vmdk and -flat.vmdk files are sought.

XSIBackup is a script...

#21 Re: General matters » Dont run from crond job » 2017-07-27 21:35:44

### 2017-08-03 Admin update ###

To trigger scripts from an outer system by using OpenSSH client use the -t SSH option to redirect output to your client.

https://33hops.com/xsibackup-remote-execution.html

### 2017-08-03 Admin update ###

Mmm, I don't know if I'm getting you right. If you want to execute a backup job (./xsibackup --backup-program --backup-type=custom...) from a remote cron:

1 - You need to make sure your remote Linux system can run SSH commands passwordlessly. Add your Linux cron system's (the system from where you want to execute the job) rsa pub key to the authorized_keys file at the ESXi server where the job is to be executed. Don't have an RSA key pair?, first of all generate one, how?, by using OpenSSL binary https://en.wikibooks.org/wiki/Cryptogra … ng_OpenSSL

2 - Run the job manually

ssh -vv -o StrictHostKeyChecking=no -i /path/to/your/key/xsibackup_id_rsa root@your.esxihost.com "/vmfs/volumes/datastore1/xsi-dir/xsibackup backup-job"

with the -vvv (verbose) option to actually see what's being sent to the ESXi host, as some characters will for sure need to be escaped.

3 - Once you have verified that the command gets to the server as it should, whithout any unescaped characters that might break its syntax coherence, add it to your root crontab: /etc/crontab at the triggering Linux box. Avoid using crontab -e and use /etc/crontab directly to be more straight and avoid security blockings.

#22 Re: General matters » connection parameters (username and password or key) » 2017-07-27 18:13:21

Thank you!

There isn't any good solution to that. It's not something about XSIBackup, you do need to write to system files to get XSIBackup up and running, and that requires root access, nothing to do here.

You also need full access to be able to run backups succesfully, as VM configuration files need to be modified to, per instance, remove disks or snapshots. You would enter a nightmare if you pretended to control your backups based on different users with different privileges as a personal project. We might offer that feature once the 10.0.0 branch is well secured and stable, but still, it might not be possible.

You cannot use Linux tools such as expect https://linux.die.net/man/1/expect to pass the password in a script cause they're simply not available. ESXi is a totally cut down environment.

That said, there's nothing that prevents you from copying your XSIBackup public key to the remote server's authorized_keys file, but that file is unique and might not be accessible to other users than root. ESXi's shell is not a regular Linux system, you can't assume you can interact with it in a "normal" way.

#23 Re: General matters » Dont run from crond job » 2017-07-27 16:32:44

We offer a working solution: xsibackup-cron, you can use it to trigger your backups and even other scripts.

You cannot assume ESXi is Linux and that things will work as you expect. ESXi is not Linux, thus thoughts as Kevin's "normal behavior", just don't apply in this case. ESXi is somewhat resembling Linux, with an extremely cut down shell and with strategically situated bugs as well as reduced functionality in every component.

I'll borrow Niki Lauda's famous quote to say: programming scripts under ESXi is like piloting a helicopter in a telephone box. So, follow our advice or put your helmet on.

I'm not saying you can't do it, there's a lot of really smart people around this forum, but fighting against ESXi's Linux false resemblance will require a lot of time and will from your part. Thus, if you are a backup professional that just wants to set thing up, I would not bother; follow our advice or even buy our PRO version to support our efforts.

#24 Re: General matters » sometimes KO, Failed to clone disk: The file specified is not a virtu » 2017-07-25 16:50:15

This looks like a hardware/ driver related problem or vCenter moving around the VMs while in the middle of a backup, but even if vMotion would move a VM to a different host, the cloning process would not be interrupted that nasty way. If you start to clone a disk and suddenly the rules of the game change in the middle of the process, or it just sometimes works, then it most probably has to do with a lower level layer.

1 - Are you using vCenter?, is vMotion active? (just in case)
2 - If the answer to any of the above questions is no, then try with a different datastore cause you're most probably hitting a hardwate/ driver issue.

Board footer