Forum ©XSIBackup: ©VMWare ©ESXi Backup Software

cdasn

Member

Registered: 2020-01-27

Posts: 12

How to check back-up command state or success

Hello,

When running a back-up command (./xsibackup --backup "VMs(RUNNING)" /vmfs/volumes/datastore2/backup-local) that is suppose to take quite a long time (like 24h, because of VMs sizes, and first backup), when my SSH terminal disconnect, I'm loosing the progress of the actual task output.

When reconnecting to my ESXi host via SSH, I'm not able to find how to check is the task is still running or not, or any error exit code.

• On free version, I used to check if an instance of XSI was still running using command

  ps | grep 'xsi'

  or I was looking at logs files.

• Right now on my DC version, ps | grep 'xsi' is empty, and logs files /var/log/error.log and /var/log/xsibackup.log are empty as well.

  On my ESXI GUI, I can see one xsi snapshot that is present.

Thanks for your help.

Version used: (c)XSIBackup-DC 1.1.0.8:

admin

Administrator

Registered: 2017-04-21

Posts: 2,057

Re: How to check back-up command state or success

If you disconnect your TTY you kill the job, unless you append the & character at the end of it, which will in turn disconnect the running process from the TTY.

Output is redirected to STDOUT when you run a backup manually. If you want the output to be logged, just redirect it to the xsibackup.log file:

./xsibackup --backup "VMs(RUNNING)" /vmfs/volumes/datastore2/backup-local >> /scratch/XSI/XSIBackup-DC/var/log/xsibackup.log 2>&1

If you have a working job in the command line, you can easily generate the job file by just appending --save-job=NNN to the job arguments

./xsibackup --backup "VMs(RUNNING)" /vmfs/volumes/datastore2/backup-local --use-smtp=1 --mail-to=me@myserver.com --save-job=001

Once you know your backup is working fine,add it to the /scratch/XSI/XSIBackup-DC/var/spool/cron/root-crontab file, like:

5    2    *    *    *   /scratch/XSI/XSIBackup-DC/etc/jobs/001 > /dev/null 2>&1

And run --update-cron

./xsibackup --update-cron

To have your schedule added to the ESXi crontab at /var/spool/cron/crontabs/root

In linux you have tools like Screen that allow you to reconnect to a disconnected TTY, in ESXi there's no such thing, thus you have to redirect output to a file to achieve the same result.

You may simply watch a running command running by using tail -f on the xsibackup.log file, like:

tail -f /scratch/XSI/XSIBackup-DC/var/log/xsibackup.log

cdasn

Member

Registered: 2020-01-27

Posts: 12

Re: How to check back-up command state or success

Thanks for your answers.

cdasn

Member

Registered: 2020-01-27

Posts: 12

Re: How to check back-up command state or success

Hello there,

1 - I tried to run that command to backup around 10 VMs.

./xsibackup --backup "VMs(RUNNING)" /vmfs/volumes/datastore2/backup-local >> /scratch/XSI/XSIBackup-DC/var/log/xsibackup.log 2>&1

When I'm checking the log file (xsibackup.log) I can see the first backup being done. If I wait a bit, I can see the progress.

Two days later, when everything is supposed to be OK, I'm going back to the log file. I'm seing output of the first, second and third backup, but nothing (nor error messages) about the rest.

Is there any way to know what happened?

2 - Is there a test command like in XSI Free?

--test-mode=true

admin

Administrator

Registered: 2017-04-21

Posts: 2,057

Re: How to check back-up command state or success

--test-mode=true in (c)XSIBackup-Free just allows to test the e-mail sending feature and also the selection of VMs nothing else.

If your backup job is not collecting the same number of VMs by using "VMs(RUNNING)" that can only be due to two circumstances:

1 - The VMs were not there. Are you using DRS?. Turn off DRS during the backup windows or use the --disable-vmotion argument.
2 - The VMs were not running. You may cherry pick the ones you want to backup by using the syntax: "VMs(VM1,VM2,VM3)"

cdasn

Member

Registered: 2020-01-27

Posts: 12

Re: How to check back-up command state or success

1 - If I'm running the command with a non existant VM, I get an error in the output file.

[root@ns3004174:/vmfs/volumes/5ba1068c-a299582c-a00f-ac1f6b006640/xsi] tail /scratch/xsi/var/log/xsibackup.log
-----------------------------------------------------------------------------------------------------------
2020-02-04T10:01:34 | Error code 1851 at file xsibackup.c, line 1851
Error description: could not find any VM in list: webserver

2 - I think I'm not using DRS, as I am not grouping ESXIs, using only free ESXIs without vSphere nor vMotion.

It looks that like even if I have no output and waited 3 days for my task to end, is has not ended.

Today when I tried to run a new command, I had the following output:

There are other --backup or --replica instances of XSIBackup-DC running
Only one instance of XSIBackup-DC client is allowed to run at a time
You can manually remove the /tmp/xsibackup.pid file, or wait.

Checking on my second host, there is a /tmp/xsibackup.pid file. So my command for backup 10vms seems to be still running after 3 days even if the output stopped.

edit: to sum up things:
on ESXI1, I ran last friday:

./xsibackup --backup "VMs(VM1,VM2,VM3,VM4,VM5,VM6,VM7,VM8,VM9,VM10)" /vmfs/volumes/datastore2/backup-local --rotate=7 >> /scratch/xsi/var/log/xsibackup.log 2>&1 &

The output stopped after VM3. Looking at datastore2, I can see folders about VM1,VM2,VM3 and that's it. The PID was still running this morning, and was killed by a new xsibackup that is used this morning.

on ESXI2, I ran last friday:

./xsibackup --backup "VMs(VM1,VM2,VM3,VM4,VM5,VM6,VM7,VM8,VM9,VM10)" /vmfs/volumes/datastore2/backup-local --rotate=7 >> /scratch/xsi/var/log/xsibackup.log 2>&1 &

The output stopped after VM3. Looking at datastore2, I can see folders about VM1,VM2,VM3 and that's it. The PID seems to be still running now.

My VMs range from 20Go to 80Go.

edit2: if I'm checking the /tmp/xsibackup.pid I get a PID number (maybe WID or CID?), and can't find any trace of it by running "ps | grep $PIDnumber" nor "ps" command.

Last edited by cdasn (2020-02-04 11:11:00)

admin

Administrator

Registered: 2017-04-21

Posts: 2,057

Re: How to check back-up command state or success

Your VMs are rather small, backup should be quite fast. Please, simplify your backup, try to backup the VM that you believe to stop the backup. Run the command on screen and post the output, we are just treating the matter on assumptions. Post the output and we'll have a clearer idea of what's going on. Nonetheless the facts you are describing point at some abnormal behaviour.

cdasn

Member

Registered: 2020-01-27

Posts: 12

Re: How to check back-up command state or success

Okay, thanks for your feedback.

You are correct, let's simplify by running on screen. Step by step.