©XSIBackup-Free: Free Backup Software for ©VMWare ©ESXi

Forum ©XSIBackup: ©VMWare ©ESXi Backup Software


Forum menu Menu

You are not logged in.

Pages: 1

#1 Re: General matters » Appear to have two running jobs at once » 2018-08-06 20:39:18

toxeia

I had previously killed all jobs before leaving the cron to take its course. After removing and reinstalling the cron again, it's still performing a double run.

My understanding is the lines in "(...)/xsi-dir/conf/root-crontab" is just inserted into the crontab located at "/var/spool/cron/crontabs/root" when ESXi boots. Is it possible there's another crontab somewhere that would be conflicting?

#2 General matters » Appear to have two running jobs at once » 2018-08-02 15:50:59

toxeia
Replies: 5

Using the cron to run a job at 0430 UTC. Checked the ESXi cron to be sure there's no doubling, but it's just the one job like in the XSIbackup cron. I receive one email stating errors in the transfer, then another email about an hour later repeating the same error. The job is running OneDiff, hot, on 3 VMs. When looking at the log, I see that it's actually connecting to the SMTP server twice. The first time it's only completed the first VM, then it performs the backup for the 2nd and 3rd before sending another email with the same error. When checking the log, I see some lines are repeated as well.

Example from the log, just from the start of the job:

2018-08-02T00:30:04|  ---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:04|  ###############################################################################
2018-08-02T00:30:04|     XSIBACKUP-PRO 11.0.2: new execution request                     
2018-08-02T00:30:04|  ###############################################################################
2018-08-02T00:30:04|  
2018-08-02T00:30:04|  ###############################################################################
2018-08-02T00:30:04|     XSIBACKUP-PRO 11.0.2: new execution request                     
2018-08-02T00:30:04|  ###############################################################################
2018-08-02T00:30:04|  
2018-08-02T00:30:04|  NOTICE: (c) XSIBackup kills any user launched jobs, make sure you don't overlap manual jobs
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:04|  NOTICE: (c) XSIBackup kills any user launched jobs, make sure you don't overlap manual jobs
---------------------------------------------------------------------------------------------------------------------------------
XSIBackup PID:          4409260                                                 HPESXi.hba.msft
Thu, 02 Aug 2018 00:30:04 +0000                                 IPv4: 192.168.2.2/255.255.255.0
VMware ESXi 6.5.0 build-8294253                              (c) Rsync 3.1.0 as opt. dependency
---------------------------------------------------------------------------------------------------------------------------------
Backup Id:                  001                        Intel(R) Xeon(R) Gold 5120 CPU @ 2.20GHz 
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:16|  Backup description: HBEsxi
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:16|  Message: crontab is installed for user root
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:17|  ADVICE: no SSD disks, please consider adding an SSD cache disk to improve performance
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:17|  Backup user is: root
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:17|  Backup program is: onediff
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:17|  Service OpenSSH ready at server 10.10.10.2:22
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:17|  Info: remote XSIBACKUP-PRO install dir has been set by means of the --remote-xsipath argument
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:18|  Remote xsi path set to: /vmfs/volumes/WDBlackR5/xsi-dir (filesystem: VMFS-6)
---------------------------------------------------------------------------------------------------------------------------------
XSIBackup PID:          4409264                                                 HPESXi.hba.msft
Thu, 02 Aug 2018 00:30:04 +0000                                 IPv4: 192.168.2.2/255.255.255.0
VMware ESXi 6.5.0 build-8294253                              (c) Rsync 3.1.0 as opt. dependency
---------------------------------------------------------------------------------------------------------------------------------
Backup Id:                  001                        Intel(R) Xeon(R) Gold 5120 CPU @ 2.20GHz 
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:19|  Backup description: HBEsxi
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:19|  Message: crontab is installed for user root
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:19|  Remote ESXi version is 6.5.0
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:19|  Mirroring to server 10.10.10.2 port 22
2018-08-02T00:30:19|  Checking Rsync exists on the other side...
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:20|  ADVICE: no SSD disks, please consider adding an SSD cache disk to improve performance
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:20|  Backup user is: root
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:20|  Backup program is: onediff
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:20|  (c) Rsync (samba.org) found at [ 10.10.10.2:22:/vmfs/volumes/WDBlackR5/xsi-dir/bin/xsibackup-rs... ]
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:20|  Service OpenSSH ready at server 10.10.10.2:22
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:20|  Info: remote XSIBACKUP-PRO install dir has been set by means of the --remote-xsipath argument
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:21|  Remote xsi path set to: /vmfs/volumes/WDBlackR5/xsi-dir (filesystem: VMFS-6)
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:21|  (c) XSIDiff found at [ 10.10.10.2:22:/vmfs/volumes/WDBlackR5/xsi-dir/bin/xsidiff... ]
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:21|  Getting list of all VMs...
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:21|  6      HBSQL        [10kSASRaid5] HBSQL/HBSQL.vmx             windows9Server64Guest   vmx-13              
2018-08-02T00:30:22|  7      HBAD         [10kSASRaid5] HBAD/HBAD.vmx               windows9Server64Guest   vmx-13              
2018-08-02T00:30:22|  Remote ESXi version is 6.5.0
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:22|  Mirroring to server 10.10.10.2 port 22
2018-08-02T00:30:22|  Checking Rsync exists on the other side...
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:23|  8      Spiceworks   [10kSASRaid5] Spiceworks/Spiceworks.vmx   windows9_64Guest        vmx-13              
2018-08-02T00:30:23|  (c) Rsync (samba.org) found at [ 10.10.10.2:22:/vmfs/volumes/WDBlackR5/xsi-dir/bin/xsibackup-rs... ]
---------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:23|  VMs to backup:
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:23|  6      HBSQL        [10kSASRaid5] HBSQL/HBSQL.vmx             windows9Server64Guest   vmx-13              
2018-08-02T00:30:24|  (c) XSIDiff found at [ 10.10.10.2:22:/vmfs/volumes/WDBlackR5/xsi-dir/bin/xsidiff... ]
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:24|  Getting list of all VMs...
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:24|  6      HBSQL        [10kSASRaid5] HBSQL/HBSQL.vmx             windows9Server64Guest   vmx-13              
2018-08-02T00:30:25|  7      HBAD         [10kSASRaid5] HBAD/HBAD.vmx               windows9Server64Guest   vmx-13              
2018-08-02T00:30:25|  8      Spiceworks   [10kSASRaid5] Spiceworks/Spiceworks.vmx   windows9_64Guest        vmx-13              
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:26|  VMs to backup:
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:26|  6      HBSQL        [10kSASRaid5] HBSQL/HBSQL.vmx             windows9Server64Guest   vmx-13              
2018-08-02T00:30:26|  7      HBAD         [10kSASRaid5] HBAD/HBAD.vmx               windows9Server64Guest   vmx-13              
2018-08-02T00:30:29|  7      HBAD         [10kSASRaid5] HBAD/HBAD.vmx               windows9Server64Guest   vmx-13              
2018-08-02T00:30:29|  8      Spiceworks   [10kSASRaid5] Spiceworks/Spiceworks.vmx   windows9_64Guest        vmx-13              
---------------------------------------------------------------------------------------------------------------------------------
2018-08-02T00:30:31|  8      Spiceworks   [10kSASRaid5] Spiceworks/Spiceworks.vmx   windows9_64Guest        vmx-13

You can see it gets the list of VMs and lists the ones ot be backed up twice - the second list of all VMs breaks up the list of VMs to be backed up.

Running the command manually doesn't create this doubling, nor does it lead to the errors. Likely my solution for now is just going to be using plink and windows scheduling to manage the backup schedule.

#3 Re: General matters » Mail fails to send upon completion » 2018-07-26 14:59:24

toxeia

I think because the documentation only mentioned Microsoft Office and Gmail I was focusing on those. I actually started experiencing issues with that relay as well, so I went with a free account at Mailgun.

A note on that though - they provide sandboxes for free, but it provides a very long subdomain consisting of a hex identifier. The user/from line ends up being 62 characters which seems to exceed what the GUI will accept. Using a text editor to edit the smtpsrvs file or inserting the smtp info into the job file directly bypasses the limitation. It also appears that the smtp-pwd entry from the gui is limited to 32 characters, and the default password provided was 50 characters. It might also be that the 33rd character is a '-' if the gui refuses special caracters.

#4 Re: © OneDiff » OneDiff fails after initial backup » 2018-07-26 14:48:54

toxeia

Just found the problem. I had been wiping the destination directory and retrying the backup, but somehow the VM was already registered on the host. Wouldn't put it past my just being careless and forgetting to unregister after testing it previously. After unregistering I was able to do subsequent backups no problem. Thank you for the assistance.

#5 © OneDiff » OneDiff fails after initial backup » 2018-07-25 21:51:07

toxeia
Replies: 2

Trying to use OneDiff to backup over the network to a second ESXi box. Finding that the first backup does fine, the certify-backup comes back green, etc. But when the second backup comes along it fails every time.

From the email after failure:

Last error raised for the above VM:
ERROR DIFCERT0, details: [Spiceworks] error: sha1 hashes are different [Spiceworks-flat.vmdk]
Complete backup elapsed time: 5 min
The ESXi configuration was saved to 10.10.10.2:22:/vmfs/volumes/WDBlackR5/XSIDest
• [ Wed Jul 25 20:43:13 UTC 2018 ] ERROR (DIFCERT0), details [Spiceworks] error: sha1 hashes are different [Spiceworks-flat.vmdk]
• [ Wed Jul 25 20:43:13 UTC 2018 ] WARNING (DIFRMVMX), details [Spiceworks] measure DIFRMVMX: took measure, renamed remote .vmx file to reinitialize OneDiff

Here is the log from the putty window during the job:

[Spiceworks] Starting backup (size is 82627M on 82921M file)
-------------------------------------------------------------------------------------------------------------------------------------
XSIBackup will backup your VMs while they are running and will quiesce guest services too, so that users
can continue to use the VM while the backup is taking place. You can also run cold and warm --backup-how
-------------------------------------------------------------------------------------------------------------------------------------
Hot backup selected for VM: [Spiceworks], will not be switched off
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] info: boot partition is MBR
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] info: Win64 OS (windows9_64) detected on MBR, applying Windows algorithm
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] info: quick size check...
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] info: file [Spiceworks-flat.vmdk]...
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] info: file size check | OK [ 85899345920 bytes | 85899345920 bytes ]
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] info: remote VMX file size 3258
-------------------------------------------------------------------------------------------------------------------------------------
System disk CIDs 830993b0|830993b0
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] (c) OneDiff algorithm
-------------------------------------------------------------------------------------------------------------------------------------
Mirror VM exists and the system disks CID are the same, starting OneDiff...
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] info: length of .vmx file before OneDiff phase is 3425
-------------------------------------------------------------------------------------------------------------------------------------
Snapshot & Quiescing
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] info: VMWare Tools detected, taking snapshot QUIESCED(false)...
[Spiceworks] info: set argument --snapshot=doquiesce to quiesce your VMs
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] info: round 2
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] info: snapshot taken, quiescing status:
-------------------------------------------------------------------------------------------------------------------------------------
Initiating OneDiff differential backup...
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] info: delta remains cleaned up at mirror VM [Spiceworks_XSIBAK]
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] info: transfering snapshot file [Spiceworks-Snapshot20.vmsn]
sending incremental file list
Spiceworks-Snapshot20.vmsn
         31,854 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=0/1)

sent 31,958 bytes  received 35 bytes  63,986.00 bytes/sec
total size is 31,854  speedup is 1.00
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] info: transfering snapshot file [Spiceworks-000001.vmdk]
sending incremental file list
Spiceworks-000001.vmdk
            339 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=0/1)

sent 439 bytes  received 35 bytes  316.00 bytes/sec
total size is 339  speedup is 0.72
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] info: transfering snapshot file [Spiceworks-000001-sesparse.vmdk]
sending incremental file list
Spiceworks-000001-sesparse.vmdk
  1,050,279,936 100%   48.71MB/s    0:00:20 (xfr#1, to-chk=0/1)

sent 1,050,536,454 bytes  received 35 bytes  48,862,162.28 bytes/sec
total size is 1,050,279,936  speedup is 1.00
-------------------------------------------------------------------------------------------------------------------------------------
3658945_8_vmx_file_tmp                                                                              100% 3265     4.1MB/s   00:00
3658945_8_vmsd_file_tmp                                                                             100%  428   610.2KB/s   00:00
-------------------------------------------------------------------------------------------------------------------------------------
Removing hot snapshot & merging data
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] info: OneDiff backup differential data (20) merged with base data
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] info: the backup mirror VM [Spiceworks_XSIBAK] Id (vim.fault.AlreadyExists) {
   faultCause = (vmodl.MethodFault) null,
   faultMessage = <unset>,
   name = "6"
   msg = "The specified key, name, or identifier '6' already exists."
} has been created
-------------------------------------------------------------------------------------------------------------------------------------
/vmfs/volumes/10kSASRaid5/xsi-dir/xsibackup: eval: line 1: syntax error: unexpected "("
[Spiceworks] info: diff mirror VM reloaded
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] info: deleting snapshot at [Spiceworks_XSIBAK] Id (vim.fault.AlreadyExists) {
   faultCause = (vmodl.MethodFault) null,
   faultMessage = <unset>,
   name = "6"
   msg = "The specified key, name, or identifier '6' already exists."
}...
/vmfs/volumes/10kSASRaid5/xsi-dir/xsibackup: eval: line 1: syntax error: unexpected "("
[Spiceworks] error DIFDELAL: error deleting all snapshots VM [Spiceworks_XSIBAK] Id (vim.fault.AlreadyExists) {
   faultCause = (vmodl.MethodFault) null,
   faultMessage = <unset>,
   name = "6"
   msg = "The specified key, name, or identifier '6' already exists."
}, details:
/vmfs/volumes/10kSASRaid5/xsi-dir/xsibackup: eval: line 6: specified: not found
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] info: Updating CID at [10.10.10.2:22:/vmfs/volumes/WDBlackR5/XSIDest/Spiceworks/Spiceworks.vmdk]
[Spiceworks] info: finished OneDiff backup, CID updated
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] info: certifying backup by means of sha1 algorithm...
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] info: computing hashes for [Spiceworks-flat.vmdk]...
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] info: sha1 hash is 30c8af01c60f7593d6acd41ff61b5deab3d0d7fd
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] error: sha1 hashes are different [Spiceworks-flat.vmdk]
-------------------------------------------------------------------------------------------------------------------------------------
[Spiceworks] measure DIFRMVMX: took measure, renamed remote .vmx file to reinitialize OneDiff
The ESXi configuration was saved to "10.10.10.2:22:/vmfs/volumes/WDBlackR5/XSIDest"

#6 Re: General matters » Mail fails to send upon completion » 2018-07-24 17:39:23

toxeia

I double checked the documentation, but I'm not seeing anything that I haven't addressed. Access for less secure applications has been enabled. I've tried removing the smtp-delay flag to allow the automatic delay to be used, but it still fails with the same error. 

The password consists only of upper/lower case letters and numbers. The username is all lowercase letters. The thing I don't understand is that I've gotten an email to send successfully, but I can't get it to happen again.

Edit:

I've come up with a solution, though it's not ideal. I'm using hMail, a Windows SMTP/POP3/IMAP server, as a SMTP Relay. I send to the local server on my network, it relays to Gmail, and I receive it in my inbox. It works with either the 2Factor account or the non 2FA that has less secure app access enabled. This will work in the meantime.

#7 General matters » Mail fails to send upon completion » 2018-07-24 16:31:49

toxeia
Replies: 4

I've experienced this with both the free and pro versions of XSIBackup. When using Gmail for the SMTP server, 9 times out of 10 the SMTP process is halted on:

220 smtp.gmail.com ESMTP y85-v6sm12493758oie.25 - gsmtp
451 4.5.0 SMTP protocol violation, see RFC 2821 y85-v6sm12493758oie.25 - gsmtp

I thought perhaps this was due to my using 2 factor for personal account (even though I was using an App password), so I created a new account without 2Fa and enabled access for less-secure applications. Issue persists.

I've tried this both using the --use-smtp command and listing out the command. Both are listed below:

2;smtp.gmail.com:465;<gmailaccount>@gmail.com;<gmailaccount>@gmail.com;<gmailpassword>;yes;TLS;1
--smtp-srv=smtp.gmail.com \
--smtp-port=465 \
--smtp-auth=yes \
--smtp-sec=TLS \
--smtp-usr=<gmailaccount>@gmail.com \
--smtp-pwd=<gmailpassword> \
--mail-from=<gmailaccount>@gmail.com \
--mail-to=<mybusinessemail@domain.com> \
--smtp-delay=1 \

I saw in the documentation that the delay was only necessary for SMTP servers that didn't support proper pipelining, but I've only gotten Gmail to send an email successfully with the 1 second delay.

Is there a better solution that Gmail that I could use? If I could run a local SMTP server I'd be fine sending the data without encryption on my local network, but I'm not sure what software I could use that would be easiest and have the smallest footprint.

Pages: 1

Board footer