#1 2017-09-07 11:37:41

prehcm
Member
Registered: 2017-09-07
Posts: 7

Need some help deciding if XSI backup is right for my needs

I'm looking for some help deciding if this product can do what I need and how best to do this. I tried figuring it out but there are so many configuration options that I am not 100% this is the right tool.

Basically I am busy with a new project which I need to ISO 27001 certify which consists of 1 vmware esxi 6.5 server which needs to be secured. So apart from securing it via firewalls, having each VM require a boot-password I am looking to secure my backups.

I assume that if I am looking to create differential possibly deduplicated backups I will need to have "local" backups and by that I mean on the same datastore where vmware and xsi backup are running and my VMs are stored (which is one and the same - its a hwardware RAID10 consisting out of 4 SSDs).
I am able to use external storage for the backups too in various forms and protocols i.e. NFS/FTP/SSH/etc.

The problem I am unsure how to solve is how to secure the server against the case in which a HD is physically stolen. Currently, if a HD is stolen, apart from it being a part of a RAID10 all you can get from it are my esxi configuration and the VMs which require a boot-password so the information is well-protected but how would I secure the backups on the HDs?

Offline

#2 2017-09-07 11:58:43

admin
Administrator
Registered: 2017-04-21
Posts: 1,366

Re: Need some help deciding if XSI backup is right for my needs

Well, in sake of speed I would backup to a different physical disk or array, if you can make it be a different datastore at the end of an NFS share better, as you will, on addition, count with some sort of physical distance between your main datastore, where the production VMs are, and your backups.

To protect backups against a physical theft, just use encryption in your backup file system.

Offline

#3 2017-09-07 12:09:45

prehcm
Member
Registered: 2017-09-07
Posts: 7

Re: Need some help deciding if XSI backup is right for my needs

admin wrote:

Well, in sake of speed I would backup to a different physical disk or array, if you can make it be a different datastore at the end of an NFS share better, as you will, on addition, count with some sort of physical distance between your main datastore, where the production VMs are, and your backups.

To protect backups against a physical theft, just use encryption in your backup file system.

Thanks for the quick reply. That makes sense so far to me. Lets assume I mount an additional datasore via a NFS share or ISCSI and the underlying HD is encrypted and only allows RW access from this server's particular IP, that sounds secure enough I assume.

If I go this route, what is the best way to do differential backups or maybe even deduplicated ones with xsi backup? Reading throug the docs on your site I see terms like OneDiff, XSITools, XSIDiff and more.
Do you have a link to a specific how-to on your blog? I see there are quite a few around but I'm not sure what I need to read up on.

Offline

#4 2017-09-07 12:17:56

admin
Administrator
Registered: 2017-04-21
Posts: 1,366

Re: Need some help deciding if XSI backup is right for my needs

If you want to perform differential backups locally, then OneDiff is your choice. It will store changed blocks in a snapshot and move only that snapshot to be merged with the previously backed up data on each backup cycle.

XSITools will copy your VMs to a deduplicated + compressed repository, thus you will need to restore your VMs prior to using them, while with OneDiff the VM will be ready to be switched on. XSITools is also differential, but blocks hashes need to be compared, so each differential chunk needs at least the time to calculate its checksum added to its backup time, whereas in OneDiff the differential blocks are kept aside and don't need to be compared against anything.

Offline

#5 2017-09-07 12:56:42

prehcm
Member
Registered: 2017-09-07
Posts: 7

Re: Need some help deciding if XSI backup is right for my needs

Thanks for the pointers but this is where things become unclear:

You say OneDiff and locally and on the other hand you say XSITools and repository so I am wondering what repository means in this case. reading up on OneDiff your site talks about datastores being the target so I assume when you say locally that means an attached datastore.

I'm now looking at OneDiff first and backing up to a datastore. To keep a historic set of backups, I found this article. but there you talk about:

Now you need a way to chain an © XSITools backup on the other end once this first stage of the process is over.

so what does "other end" mean? this is only one server with 2 datastores. Where do I execute this XSITools then?

I see the other options are i.e. borg backup. I am familiar with borg and use it in another scenario to backup a few linux servers. BUT I think to use it, I need to backup with OneDiff to a 2nd datastore then have borg running on another server which can access that datastore?


Looking closer at: XSITools: block level deduplication over VMFS - that sounds right until I read this paragraph:

Other FS available through NFS/iSCSI/FC

The above paragraph has to do with VMFS file systems, or in a more plain language, hard disks that are directly connected to our host controller and formatted by using this VMWare propietary FS. VMFS was designed to host VMs, not to store a pile of smaller chunks of data, that's why their designers weren't thinking about hosting de-duplicated data. In any case, most of the times, the storage we will be using is an external NAS connected via NFS/iSCSI/FC, therefore those datastores will be formatted under a different FS: ext3, ext4, BTRFS, XFS, NTFS, FAT32, etc...

That doesn't seem correct to me. Using ESXI 6.5 I can mount either NFS or ISCSI, format it with VMFS and use it as an additional datastore.

What I am looking for is a solution which can backup VMs to a datastore and keep a set of historic backups i.e. 4 weekly and 7 daily ones. ideally differential backups. So is XSITools what I need?

Offline

#6 2017-09-07 15:13:12

admin
Administrator
Registered: 2017-04-21
Posts: 1,366

Re: Need some help deciding if XSI backup is right for my needs

OneDiff works both: "locally", which in this case means from a locally attached datastore to another locally attached datastore and, over IP, namely, OneDiff moves the differential data from a local datastore to any other ESXi host reachable over IP. Obvioulsly to any local path attached to that remote server, but I should not need to especify that, it's implicit.

Other end means the target device, whatsoever will be receiving the data being backed up. If I use the expression other end and do not especify whether it is a local datastore or an ESXi server on the other end of the world, reachable by IP, then I'm using a vage term deliberately, because I'm talking in conceptual terms and I want to leave that option open.

If you perform a OneDiff backup to a remote server (reachable over IP), you then have a VM which has been mirrored and is an exact copy of the VM that it mirrors, and that is ready to be swicthed on, thus a usable VM from all points of view. That VM is on a different host than the production ESXi host where the VM being backed up was, and thus the referenced post, which is written in a maieutic style, states that you can then chain a backup, to be performed once the first backup finishes, that is what the term link means.

That additional backup can, and it's what the post proposes, be an XSITools backup, which will archive the just backed up VM to a repository. As the first backup which we will be using as the source of the second is in a different ESXi host, and maybe even in a different datacenter, the load associated to performing that second backup, won't interfeere with the production ESXi host.

A repository is a storage format that hosts data, like a .zip file. In this particular case, hosts the chunks of data that compose the XSITools deduplicated Virtual Machines. VMs stored in a repository cannot be used directly, as they are not in an operational state, just like a file cannot be used when it's stored in a .zip file, it first needs to be extracted, or in the case of an XSITools repository, restored.

You don't need a previous backup to backup to Borg, you can do so directly. The drawback is that XSIBackup will send all data to the Borg server. There's currently no possibility to run a Borg client in ESXi.

Yes, you can VMFS format your NFS volume, but the fact that you can does not mean you must, or that using a different File System is not correct.

https://pubs.vmware.com/vsphere-6-0/ind … e.6.9.html

Before choosing the file system you'll be using in your NFS attached device you should take into account some considerations:

- Will it be useful to use VMFS?
- Are there other File Systems you can use?
- Do you have some advantage by using a different Filesystem like ext4, btrfs, XFS, etc...?

If you want to use that NFS attached datastore to host deduplicated data, that is compounded by thousands of indexed small chunks of data, you may decide that XFS or ext4 are best choices than VMFS, as they can handle many times the amount of inodes that VMFS can.

Offline

#7 2017-09-11 08:08:22

prehcm
Member
Registered: 2017-09-07
Posts: 7

Re: Need some help deciding if XSI backup is right for my needs

Thanks for all the help, I am still reading up on what you said and trying to figure it all out. I will now go play with the free version and see what I can come up with.

One last quick question:
What I am looking for is a solution which can backup VMs (in whatever format - they don't need to be run from there I just need to be able to restore them back onto the original host) to a datastore (lets say an NFS share formatted with whatever file system you think is best) and keep a set of historic backups i.e. 4 weekly and 7 daily ones. Lets forget about deduplication for now but I need differential backups. So is XSITools what I need and is there a tutorial to point out which deals with a similar need available?

Offline

#8 2017-09-11 14:08:10

roberto
Moderator
Registered: 2017-04-22
Posts: 49

Re: Need some help deciding if XSI backup is right for my needs

Follow this tutorial: https://33hops.com/xsibackup-pro-onedif … olicy.html
If you don't want deduplication, just use Rsync to backup the OneDiff mirrored _XSIBAK.
In any case, why not using deduplication if you have it available and on top of that you already have a fresh backup?

Offline

#9 2017-09-11 14:49:46

prehcm
Member
Registered: 2017-09-07
Posts: 7

Re: Need some help deciding if XSI backup is right for my needs

Follow this tutorial: https://33hops.com/xsibackup-pro-onedif … olicy.html
In any case, why not using deduplication if you have it available and on top of that you already have a fresh backup?

if I get this right, I can't use deduplication as I stated multiple times that I do not have a second server.

I have:
1 ESXI 6.5 with 1 datastore (locally attached HDs/RAID) and 1 datastore (mounted via NFS - formatted with whatever file system is best)
So the link you gave won't work as it talks about: "chain an © XSITools backup on the other end" and I do not have another end.

I think I will demo test the PRO version and use --backup-prog=OneDiff to backup to my datastore mounted via NFS as explained above. If I get this right, this way I can limit the backup space used while using differential backups. Combien that with i.e. a weekly cron job should do the trick. Except I can't figure out how to "restore" from a OneDiff backup.

Something like:

 /vmfs/volumes/datastore1/xsi-dir/xsibackup --backup-point=/vmfs/volumes/datastore2/_backups --backup-type=all --date-dir=yes --backup-prog=OneDiff --mail-from=administrator@mydomain.tld --mail-to=administrator@mydomain.tld --smtp-srv=mail.mydomain.tld --smtp-port=25 --smtp-auth=none --smtp-usr=any --smtp-pwd=any --snapshot=includememory --backup-room=XXX

of course I also need to sort out the SMTP auth as this isn't working yet due to issues on my side.
---------------

###edit###
I realize this is going in circles and it looks like I am too blonde to use your PRO version or unserstand its usage.
I'll stick to the free version and keep 4 weekly full backups which should be enough for my current purposes.
its entirely my fault to fail to understand how to properly use the PRO version.

Thanks a lot for your tries to help me understand the product though.

Offline

Board footer