©XSIBackup-Free: Free Backup Software for ©VMWare ©ESXi

Forum ©XSIBackup: ©VMWare ©ESXi Backup Software


You are not logged in.

#2 General matters » Should --certify-backup take this long? » 2019-01-10 15:27:24

Random Thoughts
Replies: 0

Hi all,

New to the Pro version and am trying to certify backups, but as you'll see from the log, although the first backup took about 1h36m, the --certify-backup (using sha1) is still running after nearly 14h.  Is this normal?

2019-01-10T01:00:04|  ###############################################################################
2019-01-10T01:00:04|     XSIBACKUP-PRO 11.2.2: new execution request
2019-01-10T01:00:04|  ###############################################################################
2019-01-10T01:00:04|
2019-01-10T01:00:05|  NOTICE: (c) XSIBackup kills any user launched jobs, make sure you don't overlap manual jobs
---------------------------------------------------------------------------------------------------------------------------------
XSIBackup PID:           638120                                                 edge.trisect.eu
Thu, 10 Jan 2019 01:00:04 +0000                                IPv4: 94.23.36.166/255.255.255.0
VMware ESXi 6.0.0 build-3620759                              (c) Rsync 3.1.0 as opt. dependency
---------------------------------------------------------------------------------------------------------------------------------
Backup Id:                  001                 AMD Opteron(tm) Processor 4334
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:05|  Backup description: Nightly backup
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:05|  Alert: crontab is not installed for user root
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:06|  ADVICE: no SSD disks, please consider adding an SSD cache disk to improve performance
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:06|  Backup user is: root
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:06|  Backup program is: onediff
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:08|  --backup-point found at /vmfs/volumes/NAS (filesystem: NFS)
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:09|  Getting list of all VMs...
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:09|  10     Hawking          [datastore1] Hawking/Hawking.vmx   fedora64Guest   vmx-11
2019-01-10T01:00:10|  11     Kepler           [datastore1] Kepler/Kepler.vmx     fedora64Guest   vmx-11
2019-01-10T01:00:11|  12     Hubble           [datastore1] Hubble/Hubble.vmx     fedora64Guest   vmx-11
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:12|  VMs to backup:
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:12|  10     Hawking          [datastore1] Hawking/Hawking.vmx   fedora64Guest   vmx-11
2019-01-10T01:00:16|  11     Kepler           [datastore1] Kepler/Kepler.vmx     fedora64Guest   vmx-11
2019-01-10T01:00:21|  12     Hubble           [datastore1] Hubble/Hubble.vmx     fedora64Guest   vmx-11
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:25|  Calculating sizes...
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:27|  Needed room: 408 Gb.
2019-01-10T01:00:27|  Sparse size: 408 Gb.
2019-01-10T01:00:27|  Available room: 1828 Gb.
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:29|  [Hawking] Starting backup (size is 204994M on 204992M file)
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:29|  XSIBackup will backup your VMs while they are running, so that users can continue to use the VM
2019-01-10T01:00:29|  while the backup is taking place. You can also run cold and warm --backup-how
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:29|  Hot backup selected for VM: [Hawking], will not be switched off
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:30|  [Hawking] info: boot partition is MBR
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:32|  [Hawking] notice DIFRMMIS: no [Hawking-flat.vmdk] in remote OneDiff mirror, first run?.
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:32|  [Hawking] (c) OneDiff algorithm
---------------------------------------------------------------------------------------------------------------------------------
[Hawking] info: OneDiff backup first run, removing snapshots
[Hawking] info: all snapshots removed
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:53|  Snapshot & Quiescing
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:53|  [Hawking] info: VMWare Tools detected, taking snapshot QUIESCED(true)...
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:53|  [Hawking] info: round 1
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:01:05|  [Hawking] info: snapshot taken, quiescing status:
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:01:06|  INFO: found [Hawking_XSIBAK] with Id: 13
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:01:08|  Removing snapshots, please wait...
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:01:10|  Syncronizing config files
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:01:16|  Backing up virtual disks...
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:01:16|  scsi0:0.fileName = "Hawking.vmdk"
---------------------------------------------------------------------------------------------------------------------------------
Destination disk format: VMFS thin-provisioned
Cloning disk '/vmfs/volumes/datastore1/Hawking/Hawking.vmdk'...
Clone: 100% done.
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:35:51|  [Hawking] Disk copied to /vmfs/volumes/NAS/Hawking/Hawking.vmdk
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:35:53|  [Hawking] info: certifying backup by means of sha1 algorithm...
2019-01-10T01:35:54|  [Hawking] info: computing hashes for [Hawking-flat.vmdk]
2019-01-10T01:35:54|  [Hawking] info: comparing hashes of /vmfs/volumes/datastore1/Hawking/Hawking-flat.vmdk | /vmfs/volumes/NAS/Hawking/Hawking-flat.vmdk

Here's the backup job in question

"/vmfs/volumes/datastore1/xsi-dir/xsibackup" \
--backup-point=/vmfs/volumes/NAS \
--backup-prog=onediff:z \
--backup-type=all \
--certify-backup=sha1 \
--snapshot=doquiesce \
--backup-id=001 \
--description="Nightly backup" \
--mail-to=danny@trisect.uk \
--use-smtp=1 \
--exec=yes >> "/vmfs/volumes/datastore1/xsi-dir/var/logs/xsibackup.log"

#3 Re: General matters » Anomaly between 'Needed room' and 'Available room'? » 2019-01-09 17:15:57

Ok you've convinced me!!

Now running XSIbackup-Pro 11.2.2, I think it's going to take me a little while to go through all the new options and find a 'best fit' for my needs

#4 Re: General matters » Anomaly between 'Needed room' and 'Available room'? » 2019-01-07 10:30:40

Ok, first backup done with --backup-room=50, and it's......interesting!!

The backup room has been limited to 50 Gb.
Available room in device /vmfs/volumes/NAS before backup: -59 Gb.
Sparse size on disk of the selected virtual machines: 412 Gb.
Needed room in device /vmfs/volumes/NAS for backup: 412 Gb.
(Id)VM Name 	State 	Size (Gb) 	Stop 	Copy 	Start 	Time (min) 	Speed (mb/s)
(10) Hawking	ON	204/ 204	NO (hot backup)	OK	- 	46 	75/ 75
The eldest folders were deleted to make room:
Error MKROOM01: cannot make 216G of room, only 50G can be made available
/vmfs/volumes/NAS/20181230020004
/vmfs/volumes/NAS/20181231020004
/vmfs/volumes/NAS/2 0190101020003
/vmfs/volumes/NAS/20190102020003
/vmfs/volumes/NAS/20190103020003
/vmfs/volumes/NAS/20190104020004
/vmfs/volumes/NAS/20190105020004
/vmfs/volumes/NAS/20190106020004
(11) Kepler	ON	200/ 200	NO (hot backup)	OK	- 	42 	80/ 80
The eldest folders were deleted to make room:
Error MKROOM01: cannot make 212G of room, only 50G can be made available
/vmfs/volumes/NAS/20190107020003
(12) Hubble	ON	8/ 8	NO (hot backup)	OK	- 	22 	5/ 5
Available space in device /vmfs/volumes/NAS after backup: 41 Gb.
Complete backup elapsed time: 111 min
Backup of ESXi configuration is not available in XSIBACKUP-FREE
Get XSIBACKUP-PRO at http://33hops.com
USE DISCOUNT COUPON XSIDREWYI
• [ Mon Jan 7 02:08:45 UTC 2019 ] ERROR (MKROOM01), details Error: cannot make 216G of room, only 50G can be made available

• [ Mon Jan 7 02:48:27 UTC 2019 ] ERROR (MKROOM01), details Error: cannot make 212G of room, only 50G can be made available

What I read from this is that it was able to make room to back up Hawking by deleting older folders (which it did).  It then had to delete this fresh folder containing the Hawking backup to make room for the Kepler & Hubble backups.

I've now set --backup-room=250 and cron to run the backup every three hours to give me a quicker understanding of the capacity management process.

Thank you admin (whoever you are) for helping me understand the process, I guess this thread can be marked as 'solved', though I will continue to tinker for a while to find a good balance between number of backups held and storage requirements.

#5 Re: General matters » Anomaly between 'Needed room' and 'Available room'? » 2019-01-06 20:52:48

admin wrote:

if the sum of all the folders matching the mask exceeds the value set in --backup-room argument, the folders will start to be deleted starting by the eldest one. They will be removed until making enough room to fit the current backup.

If this is the case I've misunderstood --backup-room, and the information given in emails.  I'm trying a little experiment by setting --backup-room=50, each backup is roughly 14GB in size, so with the current state of the backup folder it should delete the earliest backups.

I'll post the outcome sometime tomorrow

p.s. the --del-dirs option only seems to be available in the Pro version

#6 Re: General matters » Anomaly between 'Needed room' and 'Available room'? » 2019-01-06 19:35:25

Thanks for clearing up the confusion between threads,

This is the current setting in conf/xsiopts -

datedirmask="20[1-3][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]"

These are the remote directories created by XSIbackup -

root@locker:/shared/vmware# ls -al
total 40
drwxrwsrwx+ 10 root   users 4096 Jan  6 02:02 .
drwxr-xr-x   4 root   root  4096 Jan  2 15:32 ..
drwxr-xr-x+  5 nobody users 4096 Dec 30 03:18 20181230020004
drwxr-xr-x+  5 nobody users 4096 Dec 31 03:10 20181231020004
drwxr-xr-x+  5 nobody users 4096 Jan  1 03:14 20190101020003
drwxr-xr-x+  5 nobody users 4096 Jan  2 03:16 20190102020003
drwxr-xr-x+  5 nobody users 4096 Jan  3 03:21 20190103020003
drwxr-xr-x+  5 nobody users 4096 Jan  4 03:15 20190104020004
drwxr-xr-x+  5 nobody users 4096 Jan  5 03:12 20190105020004
drwxr-xr-x+  5 nobody users 4096 Jan  6 03:15 20190106020004

As far as I can tell the datedirmask matches the created folders, I'll let it run for a few days and see what happens, unless you can see an obvious problem with what I've posted

#7 Re: General matters » Anomaly between 'Needed room' and 'Available room'? » 2019-01-04 17:15:46

Done hot backup (id: 001) using vmkfstools (no compression)
Backing up to folder /vmfs/volumes/NAS/20190104020004
The backup room has been limited to 500 Gb.
Available room in device /vmfs/volumes/NAS before backup: 404 Gb.
Sparse size on disk of the selected virtual machines: 412 Gb.
Needed room in device /vmfs/volumes/NAS for backup: 412 Gb.
(Id)VM Name 	State 	Size (Gb) 	Stop 	Copy 	Start 	Time (min) 	Speed (mb/s)
(10) Hawking	ON	204/ 204	NO (hot backup)	OK	- 	35 	97/ 97
(11) Kepler	ON	200/ 200	NO (hot backup)	OK	- 	37 	90/ 90
(12) Hubble	ON	8/ 8	NO (hot backup)	OK	- 	22 	5/ 5
Available space in device /vmfs/volumes/NAS after backup: 391 Gb.
Complete backup elapsed time: 96 min
"/vmfs/volumes/datastore1/xsi-dir/xsibackup" \
--backup-point=/vmfs/volumes/NAS \
--backup-type=all \
--date-dir=yes \
--backup-room=500 \
--snapshot=doquiesce \
--backup-id=001 \
--description="Nightly backup" \
--mail-to=danny@trisect.uk \
--use-smtp=1 \
--exec=yes >> "/vmfs/volumes/datastore1/xsi-dir/var/logs/xsibackup.log"[root@edge:/vmfs/volumes/5

#8 General matters » Anomaly between 'Needed room' and 'Available room'? » 2019-01-04 11:09:22

Random Thoughts
Replies: 10

Hi all,

I've restricted backup room to 500GB, and 'Needed room' is 412GB.  I've run a number of successful backups via a nightly cron job, but this is what I'm now seeing in my emails -

Available room in device /vmfs/volumes/NAS before backup: 404 Gb.
Sparse size on disk of the selected virtual machines: 412 Gb.
Needed room in device /vmfs/volumes/NAS for backup: 412 Gb.
Available space in device /vmfs/volumes/NAS after backup: 391 Gb.

What's happening here?  My assumption was that when 'Available room'  was less than 'Needed room', the earliest backup would be deleted to free up space.

Addendum:  I've just checked the logs and found this -

Not enough room to make the backup, some older folders will be deleted

They haven't been deleted, presumably (I'll have to check this) because the NFS share doesn't currently allow file deletion.  So am I right in thinking that XSIbackup will continue to backup VMs while it can (disk space permitting)?

Edit:  Just tried deleting the earliest backup folder manually (from SSH connection on VMWare) and it completed without any issues so I'm not sure why XSIBackup wasn't able to delete it

Thanks for looking

#9 Re: General matters » No space » 2019-01-02 23:09:41

admin wrote:

The --backup-room value is expressed in Mb., thus 690 means: use 690 Mb. to store my backups.

Are you sure?

From my backup job -

--backup-room=500

From the confirmation email (received after backup) -

The backup room has been limited to 500 Gb

#10 Re: General matters » Clarification wanted on 'Backup ESXi configuration' » 2018-12-28 10:44:25

Sorry for the late reply (Christmas got in the way)

I think the version / build of VMWare that gets reinstalled will be out of my control so I think a manual backup / restore of the configuration / VMs will be my best option.

Thanks for your detailed response

#11 General matters » Clarification wanted on 'Backup ESXi configuration' » 2018-12-16 19:37:09

Random Thoughts
Replies: 2

Hi all,

What I'd like to achieve is a state where I can reinstall VMWare in the event of a total hardware failure and be able to get back to a working state with XSIbackup Pro backups.

So I guess the question is would the backed up ESXi configuration restore the VM configurations once it's restored, so I just then need to restore each VM without any further configuration changes needed on the host.

Hope that makes sense!!

#12 General matters » Does anyone have experience of these? » 2018-11-22 11:32:40

Random Thoughts
Replies: 1

Hi,

I've been looking for remote storage for some time and it looks like SoYouStart are reluctant to replenish their stock of storage servers.  I've found these at Hetzner https://www.hetzner.com/storage/storage-box and wonder if anyone can tell me if XSIBackup can use any of the supported protocols? (BorgBackup caught my eye)

Thanks for looking

#13 Re: General matters » Unable to unzip download » 2018-03-13 16:39:24

Ok, think I misunderstood NFS, if I set things up right it will be seen as local storage to xsibackup

#14 Re: General matters » Unable to unzip download » 2018-03-13 13:44:11

Ok, not sure what happened yesterday, but I've now been able to download & install it.

I've tested email and that's working fine.  I now need to set up remote storage, can I just use any remote storage and connect via ftp / nfs / etc or does it have to be an ESXi host?

#15 General matters » Unable to unzip download » 2018-03-12 19:42:53

Random Thoughts
Replies: 4

Hi all,

The download is only 107 (KB / B?), I suspect it should be a lot bigger.

I'm getting Unzip:  Invalid zip magic error

Command used to download -

wget http://a.33hops.com/downloads/?key=<removed>

I suspect I probably need a new key now but will the same happen again?

Thanks for looking

Board footer