#1 2019-09-15 03:33:36

srwsol
Member
Registered: 2019-09-15
Posts: 7

Backup solution question

I'm playing with the free version of XSIBackup and I've got a question about how to accomplish what I want.   All I want from a backup solution is to copy every file in a VM (snapshots included) while preserving the "thin" aspect of the vmdk files, to another location, either a separate datastore attached to the host (i.e. another disk), or to a linux VM via NFS, Samba, or any other method linux supports.  I want only a single point in time full backup (i.e. I won't be doing incremental or differential backups) for offsite archive purposes.   

I tried using --backup-prog=xsitools and it appeared to do what I want, with the exception of all these extra files in a "data" directory.  From reading the documentation, if I understand this correctly, the information in the data directory is somehow used to avoid duplicate data transfers, which I'm not sure I fully understand how it works.  There was some discussion about the issue of losing some of that data causing multiple VMs or backups to become corrupt, which is what concerns me.  I want each file in the backup under the directory name of the VM to be a binary identical copy of the real file being backed up, so that I can verify the backup by using md5sum or sha1sum against the files. 

I'm just a single person consulting business and I only have a single ESXi essentials license, and I do this full archive only once a quarter (I have other methods of backing up volatile data) so that I have a reasonable starting point for my other backups if I lose my ESXi server and have to start from scratch.  That means I don't have a problem doing some of this manually just so long as I can get exactly what I want. 

Is there a way I can do this with XSIBackup?

Offline

#2 2019-09-15 19:43:17

admin
Administrator
Registered: 2017-04-21
Posts: 1,367

Re: Backup solution question

(c)XSIBackup is extremely versatile, thus you can accomplish what you want and many other things. I believe you are currently lost in the field of possibilities.

(c)XSITools is a deduplication engine, it differentially stores data in repositories with unlimited restore points. You need the Pro version to be able to restore VMs from (c)XSITools repositories though.

(c)XSIBackup-Pro offers you SHA-1 checksum verification via the --certify-backup option, which I believe will be a more cost efficient way of achieving the same.

Yes, of course. You can make exact replicas of your VMs using different methods:

--backup-prog=Vmkfstools: simple cloning using the well known Vmkfstools binary, works only in a local datastore context.
--backup-prog=Onediff: differential copy by using an intermediate snapshot, no delay, differential data is known at the moment to perform the backup, local or over IPv4. The first copy is performed via Rsync in the Free version, while the Pro version incorporates (c)XSIDiff, which allows to move data over IP being zero and NULL aware.
--backup-prog=Rsync: differential copy over IP, calculating delta data can take long, it's interesting to be used in unreliable nextworks.

Offline

#3 2019-09-15 19:50:07

srwsol
Member
Registered: 2019-09-15
Posts: 7

Re: Backup solution question

OK,  how would you recommend I use XSIBackup to achieve what I need?  I don't mind upgrading to the Pro version if that's what it takes to get it to work how I want it to.   

Are the files within the VM's directory on the backup binary identical to the actual VM files, or are they somehow tied to all this extraneous data?  If they are identical, I can just delete the data directory.

Last edited by srwsol (2019-09-15 19:53:23)

Offline

#4 2019-09-15 19:54:30

srwsol
Member
Registered: 2019-09-15
Posts: 7

Re: Backup solution question

I can't use vmfsktools because it deletes the snapshots on the backup, as I already tried that.

Last edited by srwsol (2019-09-15 19:54:50)

Offline

#5 2019-09-15 19:55:02

admin
Administrator
Registered: 2017-04-21
Posts: 1,367

Re: Backup solution question

We were writing at the same time, please, read the additional information we added before your last comment.
You can use the Free version: Vmkfstools or Rsync. If you want the checksum certification, then the Pro version will save you the time to implement it on your own.

Offline

#6 2019-09-15 19:56:08

admin
Administrator
Registered: 2017-04-21
Posts: 1,367

Re: Backup solution question

If you want to preserve snapshots, then you would need to use --backup-prog=Onediff or --backup-prog=Rsync
(c)XSIDiff wll also preserve snapshots, but the backups aren't ready to use, you would first need to restore them.

Offline

#7 2019-09-19 03:31:41

srwsol
Member
Registered: 2019-09-15
Posts: 7

Re: Backup solution question

I just tried using onediff and it did not preserve the snapshots.  I backed up using this command:

/vmfs/volumes/553f5adb-199bdfe4-29b3-001e67ca647e/xsibackup] ./xsibackup --backup-point=/vmfs/volumes/5d7c6d59-6e45dae4-813a-001e67ca647e --backup-prog=onediff --backup-type=custom --backup-vms="LH Ubuntu Server 17.04" --mail-from=zzzzzzzz@yyyyyyyy.com --mail-to=xxxxxx@yyyyyyyy.com --subject=xsibackup --test-mode=false --smtp-srv=10.172.187.253 --smtp-port=25 --smtp-auth=none --smtp-usr=xxxxxx@yyyyyyyy.com --smtp-pwd=any


Here is the source directory:

[root@intelserver:/vmfs/volumes/553f5adb-199bdfe4-29b3-001e67ca647e/VirtualMachines/LH Ubuntu Server 17.04] ls -l
total 23263240
-rw-------    1 root     root     520163328 Sep 16 02:45 LH Ubuntu Server 17.04-000001-delta.vmdk
-rw-------    1 root     root           347 Sep 15 16:27 LH Ubuntu Server 17.04-000001.vmdk
-rw-r--r--    1 root     root            53 May  1  2017 LH Ubuntu Server 17.04-60dd9dc9.hlog
-rw-------    1 root     root         28785 Sep 15 16:26 LH Ubuntu Server 17.04-Snapshot8.vmsn
-rw-r--r--    1 root     root            13 Sep 15 03:18 LH Ubuntu Server 17.04-aux.xml
-rw-------    1 root     root     34359738368 Sep 15 03:17 LH Ubuntu Server 17.04-flat.vmdk
-rw-------    1 root     root          8684 Sep 16 02:45 LH Ubuntu Server 17.04.nvram
-rw-------    1 root     root           587 Sep 15 03:17 LH Ubuntu Server 17.04.vmdk
-rw-r--r--    1 root     root         55850 Sep 14 06:35 LH Ubuntu Server 17.04.vmdk.extents
-rw-r--r--    1 root     root           433 Sep 15 16:26 LH Ubuntu Server 17.04.vmsd
-rwx------    1 root     root          3383 Sep 16 02:45 LH Ubuntu Server 17.04.vmx
-rw-------    1 root     root           150 Sep 15 03:12 LH Ubuntu Server 17.04.vmxf
-rw-------    1 root     root        399866 May 28 02:53 vmware-32.log
-rw-------    1 root     root        256628 May 30 21:19 vmware-33.log
-rw-------    1 root     root        387940 Sep 10 03:06 vmware-34.log
-rw-------    1 root     root        255566 Sep 14 02:54 vmware-35.log
-rw-------    1 root     root        256538 Sep 14 04:37 vmware-36.log
-rw-------    1 root     root        255372 Sep 14 06:13 vmware-37.log
-rw-------    1 root     root        285407 Sep 16 02:45 vmware.log

And here is the destination directory after the backup:

[root@intelserver:/vmfs/volumes/5d7c6d59-6e45dae4-813a-001e67ca647e/LH Ubuntu Server 17.04] ls -ls
total 22745096
22744064 -rw-------    1 root     root     34359738368 Sep 19 03:00 LH Ubuntu Server 17.04-flat.vmdk
  1024 -rw-------    1 root     root          8684 Sep 19 03:00 LH Ubuntu Server 17.04.nvram
     0 -rw-------    1 root     root           587 Sep 19 03:03 LH Ubuntu Server 17.04.vmdk
     0 -rw-r--r--    1 root     root           446 Sep 19 03:00 LH Ubuntu Server 17.04.vmsd
     8 -rw-r--r--    1 root     root          3383 Sep 19 03:03 LH Ubuntu Server 17.04.vmx
     0 -rw-------    1 root     root           150 Sep 19 03:00 LH Ubuntu Server 17.04.vmxf

Last edited by srwsol (2019-09-19 03:32:35)

Offline

#8 2019-09-19 12:21:11

admin
Administrator
Registered: 2017-04-21
Posts: 1,367

Re: Backup solution question

Yes, if you use Onediff in a local context, the first full copy will be made by means of Vmkfstools, this will, as you experienced, merge the existing snapshot with the base data in the target VM when performing the backup.

Why don't you just use XSIBackup-Datacenter. It's much more advanced, it's faster, it preserves every snapshot no mater where or how, it's easier to use, it allows to customize reports, it's a full binary tool, etc...

The free version has a 60GB limit, but your VM is 32GB and you say that you use it manually. We definetely recommend that you use our latest product:

https://33hops.com/xsibackup-datacenter … ackup.html

Offline

#9 2019-09-19 15:53:22

srwsol
Member
Registered: 2019-09-15
Posts: 7

Re: Backup solution question

How much does the datacenter product cost?  I couldn't find a price on the website.   I'm a single person consulting business with just one VMWare Essentials license, so price is a factor.   I have a number of VMs with over 60gb of data, so the free version won't work for me.

Offline

#10 2019-09-20 15:14:59

admin
Administrator
Registered: 2017-04-21
Posts: 1,367

Re: Backup solution question

We had some delays with the launch date, mainly because we became more ambitious and wanted to add more features to the initial commercial release. The first commercial version will be launched in some days.

Price will be 149 EUR or 169 USD, but Pro users will be given the chance to pay the difference and keep both Pro license plus DC license.

Try it out in these days and we are sure that you will find it one of the best things to spend 149 EUR in

Offline

Board footer