You are not logged in.
Pages: 1
Have a read of this, should explain it
Hi all,
New to the Pro version and am trying to certify backups, but as you'll see from the log, although the first backup took about 1h36m, the --certify-backup (using sha1) is still running after nearly 14h. Is this normal?
2019-01-10T01:00:04| ###############################################################################
2019-01-10T01:00:04| XSIBACKUP-PRO 11.2.2: new execution request
2019-01-10T01:00:04| ###############################################################################
2019-01-10T01:00:04|
2019-01-10T01:00:05| NOTICE: (c) XSIBackup kills any user launched jobs, make sure you don't overlap manual jobs
---------------------------------------------------------------------------------------------------------------------------------
XSIBackup PID: 638120 edge.trisect.eu
Thu, 10 Jan 2019 01:00:04 +0000 IPv4: 94.23.36.166/255.255.255.0
VMware ESXi 6.0.0 build-3620759 (c) Rsync 3.1.0 as opt. dependency
---------------------------------------------------------------------------------------------------------------------------------
Backup Id: 001 AMD Opteron(tm) Processor 4334
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:05| Backup description: Nightly backup
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:05| Alert: crontab is not installed for user root
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:06| ADVICE: no SSD disks, please consider adding an SSD cache disk to improve performance
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:06| Backup user is: root
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:06| Backup program is: onediff
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:08| --backup-point found at /vmfs/volumes/NAS (filesystem: NFS)
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:09| Getting list of all VMs...
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:09| 10 Hawking [datastore1] Hawking/Hawking.vmx fedora64Guest vmx-11
2019-01-10T01:00:10| 11 Kepler [datastore1] Kepler/Kepler.vmx fedora64Guest vmx-11
2019-01-10T01:00:11| 12 Hubble [datastore1] Hubble/Hubble.vmx fedora64Guest vmx-11
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:12| VMs to backup:
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:12| 10 Hawking [datastore1] Hawking/Hawking.vmx fedora64Guest vmx-11
2019-01-10T01:00:16| 11 Kepler [datastore1] Kepler/Kepler.vmx fedora64Guest vmx-11
2019-01-10T01:00:21| 12 Hubble [datastore1] Hubble/Hubble.vmx fedora64Guest vmx-11
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:25| Calculating sizes...
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:27| Needed room: 408 Gb.
2019-01-10T01:00:27| Sparse size: 408 Gb.
2019-01-10T01:00:27| Available room: 1828 Gb.
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:29| [Hawking] Starting backup (size is 204994M on 204992M file)
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:29| XSIBackup will backup your VMs while they are running, so that users can continue to use the VM
2019-01-10T01:00:29| while the backup is taking place. You can also run cold and warm --backup-how
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:29| Hot backup selected for VM: [Hawking], will not be switched off
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:30| [Hawking] info: boot partition is MBR
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:32| [Hawking] notice DIFRMMIS: no [Hawking-flat.vmdk] in remote OneDiff mirror, first run?.
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:32| [Hawking] (c) OneDiff algorithm
---------------------------------------------------------------------------------------------------------------------------------
[Hawking] info: OneDiff backup first run, removing snapshots
[Hawking] info: all snapshots removed
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:53| Snapshot & Quiescing
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:53| [Hawking] info: VMWare Tools detected, taking snapshot QUIESCED(true)...
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:00:53| [Hawking] info: round 1
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:01:05| [Hawking] info: snapshot taken, quiescing status:
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:01:06| INFO: found [Hawking_XSIBAK] with Id: 13
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:01:08| Removing snapshots, please wait...
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:01:10| Syncronizing config files
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:01:16| Backing up virtual disks...
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:01:16| scsi0:0.fileName = "Hawking.vmdk"
---------------------------------------------------------------------------------------------------------------------------------
Destination disk format: VMFS thin-provisioned
Cloning disk '/vmfs/volumes/datastore1/Hawking/Hawking.vmdk'...
Clone: 100% done.
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:35:51| [Hawking] Disk copied to /vmfs/volumes/NAS/Hawking/Hawking.vmdk
---------------------------------------------------------------------------------------------------------------------------------
2019-01-10T01:35:53| [Hawking] info: certifying backup by means of sha1 algorithm...
2019-01-10T01:35:54| [Hawking] info: computing hashes for [Hawking-flat.vmdk]
2019-01-10T01:35:54| [Hawking] info: comparing hashes of /vmfs/volumes/datastore1/Hawking/Hawking-flat.vmdk | /vmfs/volumes/NAS/Hawking/Hawking-flat.vmdk
Here's the backup job in question
"/vmfs/volumes/datastore1/xsi-dir/xsibackup" \
--backup-point=/vmfs/volumes/NAS \
--backup-prog=onediff:z \
--backup-type=all \
--certify-backup=sha1 \
--snapshot=doquiesce \
--backup-id=001 \
--description="Nightly backup" \
--mail-to=danny@trisect.uk \
--use-smtp=1 \
--exec=yes >> "/vmfs/volumes/datastore1/xsi-dir/var/logs/xsibackup.log"
Ok you've convinced me!!
Now running XSIbackup-Pro 11.2.2, I think it's going to take me a little while to go through all the new options and find a 'best fit' for my needs
Ok, first backup done with --backup-room=50, and it's......interesting!!
The backup room has been limited to 50 Gb.
Available room in device /vmfs/volumes/NAS before backup: -59 Gb.
Sparse size on disk of the selected virtual machines: 412 Gb.
Needed room in device /vmfs/volumes/NAS for backup: 412 Gb.
(Id)VM Name State Size (Gb) Stop Copy Start Time (min) Speed (mb/s)
(10) Hawking ON 204/ 204 NO (hot backup) OK - 46 75/ 75
The eldest folders were deleted to make room:
Error MKROOM01: cannot make 216G of room, only 50G can be made available
/vmfs/volumes/NAS/20181230020004
/vmfs/volumes/NAS/20181231020004
/vmfs/volumes/NAS/2 0190101020003
/vmfs/volumes/NAS/20190102020003
/vmfs/volumes/NAS/20190103020003
/vmfs/volumes/NAS/20190104020004
/vmfs/volumes/NAS/20190105020004
/vmfs/volumes/NAS/20190106020004
(11) Kepler ON 200/ 200 NO (hot backup) OK - 42 80/ 80
The eldest folders were deleted to make room:
Error MKROOM01: cannot make 212G of room, only 50G can be made available
/vmfs/volumes/NAS/20190107020003
(12) Hubble ON 8/ 8 NO (hot backup) OK - 22 5/ 5
Available space in device /vmfs/volumes/NAS after backup: 41 Gb.
Complete backup elapsed time: 111 min
Backup of ESXi configuration is not available in XSIBACKUP-FREE
Get XSIBACKUP-PRO at https://33hops.com
USE DISCOUNT COUPON XSIDREWYI
• [ Mon Jan 7 02:08:45 UTC 2019 ] ERROR (MKROOM01), details Error: cannot make 216G of room, only 50G can be made available
• [ Mon Jan 7 02:48:27 UTC 2019 ] ERROR (MKROOM01), details Error: cannot make 212G of room, only 50G can be made available
What I read from this is that it was able to make room to back up Hawking by deleting older folders (which it did). It then had to delete this fresh folder containing the Hawking backup to make room for the Kepler & Hubble backups.
I've now set --backup-room=250 and cron to run the backup every three hours to give me a quicker understanding of the capacity management process.
Thank you admin (whoever you are) for helping me understand the process, I guess this thread can be marked as 'solved', though I will continue to tinker for a while to find a good balance between number of backups held and storage requirements.
if the sum of all the folders matching the mask exceeds the value set in --backup-room argument, the folders will start to be deleted starting by the eldest one. They will be removed until making enough room to fit the current backup.
If this is the case I've misunderstood --backup-room, and the information given in emails. I'm trying a little experiment by setting --backup-room=50, each backup is roughly 14GB in size, so with the current state of the backup folder it should delete the earliest backups.
I'll post the outcome sometime tomorrow
p.s. the --del-dirs option only seems to be available in the Pro version
Thanks for clearing up the confusion between threads,
This is the current setting in conf/xsiopts -
datedirmask="20[1-3][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]"
These are the remote directories created by XSIbackup -
root@locker:/shared/vmware# ls -al
total 40
drwxrwsrwx+ 10 root users 4096 Jan 6 02:02 .
drwxr-xr-x 4 root root 4096 Jan 2 15:32 ..
drwxr-xr-x+ 5 nobody users 4096 Dec 30 03:18 20181230020004
drwxr-xr-x+ 5 nobody users 4096 Dec 31 03:10 20181231020004
drwxr-xr-x+ 5 nobody users 4096 Jan 1 03:14 20190101020003
drwxr-xr-x+ 5 nobody users 4096 Jan 2 03:16 20190102020003
drwxr-xr-x+ 5 nobody users 4096 Jan 3 03:21 20190103020003
drwxr-xr-x+ 5 nobody users 4096 Jan 4 03:15 20190104020004
drwxr-xr-x+ 5 nobody users 4096 Jan 5 03:12 20190105020004
drwxr-xr-x+ 5 nobody users 4096 Jan 6 03:15 20190106020004
As far as I can tell the datedirmask matches the created folders, I'll let it run for a few days and see what happens, unless you can see an obvious problem with what I've posted
Done hot backup (id: 001) using vmkfstools (no compression)
Backing up to folder /vmfs/volumes/NAS/20190104020004
The backup room has been limited to 500 Gb.
Available room in device /vmfs/volumes/NAS before backup: 404 Gb.
Sparse size on disk of the selected virtual machines: 412 Gb.
Needed room in device /vmfs/volumes/NAS for backup: 412 Gb.
(Id)VM Name State Size (Gb) Stop Copy Start Time (min) Speed (mb/s)
(10) Hawking ON 204/ 204 NO (hot backup) OK - 35 97/ 97
(11) Kepler ON 200/ 200 NO (hot backup) OK - 37 90/ 90
(12) Hubble ON 8/ 8 NO (hot backup) OK - 22 5/ 5
Available space in device /vmfs/volumes/NAS after backup: 391 Gb.
Complete backup elapsed time: 96 min
"/vmfs/volumes/datastore1/xsi-dir/xsibackup" \
--backup-point=/vmfs/volumes/NAS \
--backup-type=all \
--date-dir=yes \
--backup-room=500 \
--snapshot=doquiesce \
--backup-id=001 \
--description="Nightly backup" \
--mail-to=danny@trisect.uk \
--use-smtp=1 \
--exec=yes >> "/vmfs/volumes/datastore1/xsi-dir/var/logs/xsibackup.log"[root@edge:/vmfs/volumes/5
Hi all,
I've restricted backup room to 500GB, and 'Needed room' is 412GB. I've run a number of successful backups via a nightly cron job, but this is what I'm now seeing in my emails -
Available room in device /vmfs/volumes/NAS before backup: 404 Gb.
Sparse size on disk of the selected virtual machines: 412 Gb.
Needed room in device /vmfs/volumes/NAS for backup: 412 Gb.
Available space in device /vmfs/volumes/NAS after backup: 391 Gb.
What's happening here? My assumption was that when 'Available room' was less than 'Needed room', the earliest backup would be deleted to free up space.
Addendum: I've just checked the logs and found this -
Not enough room to make the backup, some older folders will be deleted
They haven't been deleted, presumably (I'll have to check this) because the NFS share doesn't currently allow file deletion. So am I right in thinking that XSIbackup will continue to backup VMs while it can (disk space permitting)?
Edit: Just tried deleting the earliest backup folder manually (from SSH connection on VMWare) and it completed without any issues so I'm not sure why XSIBackup wasn't able to delete it
Thanks for looking
The --backup-room value is expressed in Mb., thus 690 means: use 690 Mb. to store my backups.
Are you sure?
From my backup job -
--backup-room=500
From the confirmation email (received after backup) -
The backup room has been limited to 500 Gb
Sorry for the late reply (Christmas got in the way)
I think the version / build of VMWare that gets reinstalled will be out of my control so I think a manual backup / restore of the configuration / VMs will be my best option.
Thanks for your detailed response
Hi all,
What I'd like to achieve is a state where I can reinstall VMWare in the event of a total hardware failure and be able to get back to a working state with XSIbackup Pro backups.
So I guess the question is would the backed up ESXi configuration restore the VM configurations once it's restored, so I just then need to restore each VM without any further configuration changes needed on the host.
Hope that makes sense!!
Hi,
I've been looking for remote storage for some time and it looks like SoYouStart are reluctant to replenish their stock of storage servers. I've found these at Hetzner https://www.hetzner.com/storage/storage-box and wonder if anyone can tell me if XSIBackup can use any of the supported protocols? (BorgBackup caught my eye)
Thanks for looking
Ok, think I misunderstood NFS, if I set things up right it will be seen as local storage to xsibackup
Ok, not sure what happened yesterday, but I've now been able to download & install it.
I've tested email and that's working fine. I now need to set up remote storage, can I just use any remote storage and connect via ftp / nfs / etc or does it have to be an ESXi host?
Hi all,
The download is only 107 (KB / B?), I suspect it should be a lot bigger.
I'm getting Unzip: Invalid zip magic error
Command used to download -
wget /?key=<removed>
I suspect I probably need a new key now but will the same happen again?
Thanks for looking
Pages: 1