Posts by vscheuber

vscheuber · General matters

Thanks, admin! Has anyone ever come across the loss of all their snapshots? I don't want to post my full backup logs here, but would you be willing to take a quick look and tell me if you can spot anything that would remove all the snapshots from my original VMs?

I do see sporadic error messages concerning snapshots, like this one:

---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:30:45|  [VM1] Starting backup (size is 34937M on 58362M file)
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:30:45|  XSIBackup will backup your VMs while they are running, so that users can continue to use the VM
2019-11-20T08:30:45|  while the backup is taking place. You can also run --backup-how=cold|warm
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:30:45|  Hot backup selected for VM: [VM1], will not be switched off
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:30:45|  [VM1] info: boot partition is MBR
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:30:46|  2      vMac                            [datastore10] vMac/vMac.vmx                                                     darwin16_64Guest        vmx-13                  
2019-11-20T08:30:47|  [VM1] info: quick size check...
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:30:47|  [VM1] info: file [FRDP-flat.vmdk]...
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:30:47|  [VM1] info: file size check | OK [ 59990000640 bytes | 59990000640 bytes ]
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:30:47|  [VM1] info: remote VMX file size 2858
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:30:47|  System disk CIDs 11c8d76f|11c8d76f
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:30:47|  [VM1] (c) OneDiff algorithm
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:30:47|  Mirror VM exists and the system disks CID are the same, starting OneDiff...
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:30:47|  [VM1] info: length of .vmx file before OneDiff phase is 3018
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:30:48|  Snapshot & Quiescing
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:30:48|  [VM1] info: VMWare Tools detected, taking snapshot QUIESCED(false)...
2019-11-20T08:30:48|  [VM1] info: set argument --snapshot=doquiesce to quiesce your VMs
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:30:48|  [VM1] info: round 4
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:30:51|  20     Builder                         [datastore3] Builder/Builder.vmx                                               centos7_64Guest         vmx-14                  
2019-11-20T08:30:54|  [VM1] info: snapshot taken, quiescing status: 
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:30:54|  4      Windows 10 Desktop              [datastore10] Windows 10 Desktop/Windows 8.1 Desktop.vmx                        windows9_64Guest        vmx-11                  
2019-11-20T08:30:54|  INFO: found [VM1_XSIBAK] with Id: 21
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:30:55|  Initiating OneDiff differential backup...
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:30:55|  [VM1] info: delta remains cleaned up at mirror VM [VM1_XSIBAK]
---------------------------------------------------------------------------------------------------------------------------------
[VM1] info: transfering snapshot file [FRDP_36-Snapshot20.vmsn]
[32m(c) XSIDiff 1.1.1.4 licensed: 
[0mA full file copy will be performed, no meta-data available
Reading from file /vmfs/volumes/datastore10/FRDP_36/FRDP_36-Snapshot20.vmsn
Writing to file /vmfs/volumes/datastore11/matterhorn/VM1/FRDP_36-Snapshot20.vmsn
Set input file offset to 0 bytes

100.00% done
Time taken: 1 seconds
Speed 1.00 mb/s
Overall speed 0.03 mb/s
---------------------------------------------------------------------------------------------------------------------------------
[VM1] info: transfering snapshot file [FRDP-000001.vmdk]
[32m(c) XSIDiff 1.1.1.4 licensed: 
[0mA full file copy will be performed, no meta-data available
Reading from file /vmfs/volumes/datastore10/FRDP_36/FRDP-000001.vmdk
Writing to file /vmfs/volumes/datastore11/matterhorn/VM1/FRDP-000001.vmdk
Set input file offset to 0 bytes

100.00% done
Time taken: 1 seconds
Speed 1.00 mb/s
Overall speed 0.00 mb/s
---------------------------------------------------------------------------------------------------------------------------------
[VM1] info: transfering snapshot file [FRDP-000001-delta.vmdk]
[32m(c) XSIDiff 1.1.1.4 licensed: 
[0mA full file copy will be performed, no meta-data available
Reading from file /vmfs/volumes/datastore10/FRDP_36/FRDP-000001-delta.vmdk
Writing to file /vmfs/volumes/datastore11/matterhorn/VM1/FRDP-000001-delta.vmdk
Set input file offset to 0 bytes

0.09% done
...
100.00% done
2019-11-20T08:31:14|  Needed room: 1501 Gb.
2019-11-20T08:31:14|  Sparse size: 1787 Gb.

Time taken: 18 seconds
Speed 64.06 mb/s
Overall speed 64.01 mb/s
2019-11-20T08:31:14|  Available room: 13934 Gb.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------
2019-11-20T08:31:15|  [VM1] info: length of remote .vmx file after copying it is 2857
2019-11-20T08:31:15|  [VM1] info: length of remote .vmx after replacing name is 2864
2019-11-20T08:31:15|  [VM1] info: name changed to [VM1_XSIBAK]
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:31:15|  Removing hot snapshot & merging data
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:31:16|  [VM1] Starting backup (size is 34937M on 58362M file)
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:31:16|  XSIBackup will backup your VMs while they are running, so that users can continue to use the VM
2019-11-20T08:31:16|  while the backup is taking place. You can also run --backup-how=cold|warm
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:31:16|  Hot backup selected for VM: [VM1], will not be switched off
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:31:16|  [VM1] info: boot partition is MBR
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:31:17|  Removing snapshots, please wait...
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:31:19|  Syncronizing config files
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:31:22|  Memory will be included in the snapshot
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:31:22|  [VM1] info: VMWare Tools detected, taking snapshot QUIESCED(false)...
2019-11-20T08:31:22|  [VM1] info: set argument --snapshot=doquiesce to quiesce your VMs
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:31:25|  [VM1] error CLCRESN1: cannot create snapshot, details: Create Snapshot:
2019-11-20T08:31:25|  Create snapshot failed
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:31:26|  Backing up virtual disks...
---------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:31:40|  [VM2] Starting backup (size is 179202M on 179190M file)
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:31:40|  XSIBackup will backup your VMs while they are running, so that users can continue to use the VM
2019-11-20T08:31:40|  while the backup is taking place. You can also run --backup-how=cold|warm
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:31:40|  Hot backup selected for VM: [VM2], the VM is off though
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:31:41|  [VM2] info: boot partition is MBR
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:31:41|  [VM2] info: Win64 OS (windows8Server64) detected on MBR, applying Windows algorithm
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:31:42|  Removing snapshots, please wait...
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:31:42|  Syncronizing config files
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:31:45|  Backing up virtual disks...
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:31:45|  scsi0:0.fileName = "VM2-000004.vmdk"
---------------------------------------------------------------------------------------------------------------------------------
Destination disk format: VMFS thin-provisioned
Cloning disk '/vmfs/volumes/datastore10/VM2/VM2-000004.vmdk'...

Clone: 6% done.2019-11-20T08:34:19|  [VM1] info: OneDiff backup differential data (20) merged with base data
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:34:20|  [VM1] info: diff mirror VM reloaded
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:34:20|  [VM1] info: deleting snapshot at [VM1_XSIBAK] Id 21...
2019-11-20T08:34:21|  [VM1] info: all snapshots removed
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:34:21|  [VM1] info: Updating CID at [/vmfs/volumes/datastore11/matterhorn/VM1/FRDP.vmdk]
2019-11-20T08:34:21|  [VM1] info: finished OneDiff backup, CID updated
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:34:21|  [VM1] info: performing quick check...
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:34:21|  [VM1] info: file [FRDP-flat.vmdk]...
2019-11-20T08:34:21|  [VM1] error: different sizes for [FRDP-flat.vmdk] [ 59990000640 <ne>  ]
2019-11-20T08:34:21|  [VM1] error DIFQCSIZ: file size check KO...
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:34:21|  [VM1] info: comparing first 500M of [FRDP-flat.vmdk]...
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:34:24|  [VM1] error DIFQMSH2: empty first 500M remote (inexistent file?)...
2019-11-20T08:34:24|  [VM1] info: (source) first 500M hash is: 39c15b3ad04137cda1756bd2b6cadc59a8f0379a
2019-11-20T08:34:24|  [VM1] info: (target) first 500M hash is: da39a3ee5e6b4b0d3255bfef95601890afd80709
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:34:24|  [VM1] error DIFQMSH4: first 500M mistmatch...
---------------------------------------------------------------------------------------------------------------------------------
2019-11-20T08:34:24|  [VM1] measure DIFRMVMX: took measure, renamed remote .vmx file to reinitialize OneDiff

Looking at the dates of the errors, this might have well happened while I was debugging this issue: DOW crontab entries execute daily and had the full backup and the incremental backup jobs run at the same time. There were no more errors after that. Could the two jobs run in parallel have cause the loss of snapshots on the original VMs?

vscheuber · General matters

I think my original statement was unclear. I did not expect snapshots to survive in the backups. My report is about all my snapshots in the original VMs being lost. So, this is what's happened:

Before:

Original                    Backup
- VM1                       - VM1
  - Snapshot1
  - Snapshot2            
- VM2                       - VM2
  - Snapshot1

Now:

Original                    Backup
- VM1                       - VM1           
- VM2                       - VM2

I never expected snapshots to make it into my backups. But I did not expect to lose my original snapshots in my running VMs.

Any comments on the duplication of VMs? I did not import VMs from the backup location. Is that a side effect of XSIBackup? A malfunction?

vscheuber · General matters

Hi,

I'm feeling a bit unsure asking this but I have been very happy with XSIBackup Pro until I discovered this morning that all my manual snapshots of my VMs had disappeared. I have XSIBackup Pro setup to take full a backup of all VMs weekly and incremental backups of running VMs daily. I honestly can't remember whether I validated that snapshots were still there when I configured XSIBackup initially and whether I am just missing something that's logical to everybody else (that snapshots will be removed). So my question to you all here is: Is what I am seeing normal?

I also noticed that the number of VMs has doubled. It appears as if XSIBackup has duplicated all my VMs, the duplicates being named <original name>_XSIBAK and they seem to link to the files in my backup location.

VM Duplication

I don't remember seeing any errors reported in any of the daily job emails, so I am not sure what happened and where to look. Here are the two job definitions I am using:

001 - Full Backup

[root@esxi:/vmfs/volumes/f8282ebf-45634a14/xsi-dir/jobs] cat 001
"/vmfs/volumes/datastore11/xsi-dir/xsibackup" \
--backup-prog=Vmkfstools \
--snapshot=includememory \
--backup-point=/vmfs/volumes/datastore11/matterhorn/full \
--backup-type=All \
--backup-how=Hot \
--use-smtp=1 \
--mail-to=name@gmail.com \
--backup-id=001 \
--description="All VMs" \
--date-dir=yes \
--backup-room=5000 \
--exec=yes >> "/vmfs/volumes/datastore11/xsi-dir/var/logs/xsibackup.log"

002 - Incremental Backup

[root@esxi:/vmfs/volumes/f8282ebf-45634a14/xsi-dir/jobs] cat 002
"/vmfs/volumes/datastore11/xsi-dir/xsibackup" \
--backup-prog=OneDiff \
--snapshot=includememory \
--backup-point=/vmfs/volumes/datastore11/matterhorn/incremental \
--backup-type=Running \
--backup-how=Hot \
--use-smtp=1 \
--mail-to=name@gmail.com \
--backup-id=002 \
--description="Running VMs" \
--exec=yes >> "/vmfs/volumes/datastore11/xsi-dir/var/logs/xsibackup.log"

Thank you for your insights!

Cheers.
Volker

vscheuber · General matters

@lmasarati I know this is an old thread but still wanted to inquire if you ended up figuring this out? I just discovered this thread after creating a very similar one myself. I believe I found the solution. Here is the new thread:

https://33hops.com/forum/viewtopic.php?id=609

vscheuber · General matters

@Admin, I think it would help all of us newbies if on this page the tool would call out that "7" should be avoided for the DOW:

┌───────────────────────────────────────────────────────────────────────────────────────┐
│ Enter the values for the cron schedule, as you would do in any crontab, i.e.:         │
│                                                                                       │
│ Example 1: 30   2   *   *   *        (daily job at 02:30 am)                          │
│ Example 2: 10   14   *   *   1,3,5   (job at 14:10 on M, X and F)                     │
│ Example 3: 47   8   1,15   *   *     (job at 08:47 every 1st & 15th day of month)     │
│                                                                                       │
│ Be careful not to overlap jobs by setting them too close in time                      │
│ Use job chains to ensure overlap does not occur when combining them                   │
│ Set all time fields to asterisk (*) to remove the job from the cron                   │
│                                                                                       │
│ Current time is: Fri Nov 22 20:30:10 UTC 2019                                         │
│ ┌───────────────────────────────────────────────────────────────────────────────────┐ │
│ │ Minute     Hour       Day        Month      Weekday    BackupId                   │ │
│ │                                                                                   │ │
│ │ 30         8          *          *          0          001                        │ │
│ │                                                                                   │ │
│ └───────────────────────────────────────────────────────────────────────────────────┘ │
├───────────────────────────────────────────────────────────────────────────────────────┤
│                           <  OK  >              <Cancel>                              │
└───────────────────────────────────────────────────────────────────────────────────────┘

I know this is not an XSIBackup limitation but since your software relies on the crontab provided by ESXi, this callout might save others shave hours or days off the learning curve.

Cheers!

vscheuber · General matters

OK, I researched this further and it appears as if busybox cron doesn't support "7" for Sunday, only "0". At least that's what this thread indicates:

https://communities.vmware.com/thread/342106

So I will change my crontab to use "0" instead of "7" for Sunday and hopefully my jobs will no longer overlap.

vscheuber · General matters

Here is the log entry, they both start at the same time and of course interfere with each other, so both complete with errors:

2019-11-22T08:30:06|  ###############################################################################
2019-11-22T08:30:06|     XSIBACKUP-PRO 11.2.10: new execution request                     
2019-11-22T08:30:06|  ###############################################################################
2019-11-22T08:30:06|  
2019-11-22T08:30:06|  ###############################################################################
2019-11-22T08:30:06|     XSIBACKUP-PRO 11.2.10: new execution request                     
2019-11-22T08:30:06|  ###############################################################################
2019-11-22T08:30:06|  
2019-11-22T08:30:06|  NOTICE: (c) XSIBackup kills any user launched jobs, make sure you don't overlap manual jobs
---------------------------------------------------------------------------------------------------------------------------------
2019-11-22T08:30:06|  NOTICE: (c) XSIBackup kills any user launched jobs, make sure you don't overlap manual jobs
---------------------------------------------------------------------------------------------------------------------------------
XSIBackup PID:          2340148                               matterhorn.yyy.zzz
Fri, 22 Nov 2019 08:30:06 +0000                                IPv4: aaa.bbb.ccc.ddd/255.255.255.0
VMware ESXi 6.7.0 build-10302608                              (c) Rsync 3.1.0 as opt. dependency
---------------------------------------------------------------------------------------------------------------------------------
Backup Id:                  002XSIBackup PID:          2340147                               matterhorn.yyy.zzz
Fri, 22 Nov 2019 08:30:06 +0000                                IPv4: aaa.bbb.ccc.ddd/255.255.255.0
VMware ESXi 6.7.0 build-10302608                              (c) Rsync 3.1.0 as opt. dependency
---------------------------------------------------------------------------------------------------------------------------------
Backup Id:                  001                        Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz 
---------------------------------------------------------------------------                        Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz 
----------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------

vscheuber · General matters

Hi,

I'm using XSIBACKUP-PRO 11.2.10 and the crontab file below to schedule 2 backup jobs:

001: Full backup of all VMs using Vmkfstools every Sunday
002: Incremental backup of all running VMs using OneDiff Monday through Saturday

For some reason, both jobs run every day and I don't understand why? Neither of the jobs is chained to the other. I hope for a simple user error on my side which a few extra pair of eyes can help me pinpoint.

These are my job configs and crontab:

001:

[root@matterhorn:~] cat /vmfs/volumes/datastore5/xsi-dir/jobs/001
"/vmfs/volumes/datastore5/xsi-dir/xsibackup" \
--backup-prog=Vmkfstools \
--snapshot=includememory \
--backup-point=/vmfs/volumes/datastore5/matterhorn/full \
--backup-type=All \
--backup-how=Hot \
--use-smtp=1 \
--mail-to=vscheuber@gmail.com \
--backup-id=001 \
--description="All VMs" \
--date-dir=yes \
--backup-room=5000 \
--exec=yes >> "/vmfs/volumes/datastore5/xsi-dir/var/logs/xsibackup.log"

002:

[root@matterhorn:~] cat /vmfs/volumes/datastore5/xsi-dir/jobs/002
"/vmfs/volumes/datastore5/xsi-dir/xsibackup" \
--backup-prog=OneDiff \
--snapshot=includememory \
--backup-point=/vmfs/volumes/datastore5/matterhorn/incremental \
--backup-type=Running \
--backup-how=Hot \
--use-smtp=1 \
--mail-to=vscheuber@gmail.com \
--backup-id=002 \
--description="Running VMs" \
--exec=yes >> "/vmfs/volumes/datastore5/xsi-dir/var/logs/xsibackup.log"

Crontab:

[root@matterhorn:~] cat /var/spool/cron/crontabs/root 
#min hour day mon dow command
1    1    *   *   *   /sbin/tmpwatch.py
1    *    *   *   *   /sbin/auto-backup.sh
0    *    *   *   *   /usr/lib/vmware/vmksummary/log-heartbeat.py
*/5  *    *   *   *   /bin/hostd-probe.sh ++group=host/vim/vmvisor/hostd-probe/stats/sh
00   1    *   *   *   localcli storage core device purge
30 8 * * 1-6 "/vmfs/volumes/datastore5/xsi-dir/jobs/002" # Added by XSIBackup
30 8 * * 7 "/vmfs/volumes/datastore5/xsi-dir/jobs/001" # Added by XSIBackup

Does anything jump the eye as to why both my jobs execute?

Thank y'all!

vscheuber · General matters

@JackiMode, did @admin's advice help?

From your command it appears to me you were already running NFS 3 or the command wouldn't have returned any response. At least on my ESXi 6.7u1 my NFS 4.1 share on my Synology NAS only shows up when using "esxcli storage nfs41 list". When I use "esxcli storage nfs list" nothing is returned.

I'm wondering because I have backup jobs getting stuck for no apparent reason and if NFS 4.1 is really the culprit, that would be an easy fix. Just looking for some confirmation before I make the switch.

vscheuber · General matters

@zipsonic, try:

export TERM=xterm

before launching xsibackup. This should remedy the issue as noted here:
https://33hops.com/xsibackup-pro-gui-manual.html#1

Forum ©XSIBackup: ©VMWare ©ESXi Backup Software

#1 Re: General matters » xsibackup removing snapshots? » 2020-01-08 18:20:54

#2 Re: General matters » xsibackup removing snapshots? » 2020-01-08 15:13:05

#3 General matters » xsibackup removing snapshots? » 2020-01-08 02:57:05

#4 Re: General matters » cron jobs running on wrong day? » 2019-11-22 23:50:55

#5 Re: General matters » DOW crontab entries execute daily » 2019-11-22 20:38:02

#6 Re: General matters » DOW crontab entries execute daily » 2019-11-22 19:23:27

#7 Re: General matters » DOW crontab entries execute daily » 2019-11-22 16:53:40

#8 General matters » DOW crontab entries execute daily » 2019-11-22 16:01:23

#9 Re: General matters » Problem with nfs share from QNAP » 2019-11-21 01:06:53

#10 Re: General matters » Running xsibackup results in clearing the screen and exiting. No GUI » 2019-11-21 00:55:57

Board footer