Registered users
Linkedin Twitter Google+
Close

In order to improve user's experience and to enable some functionalities by tracking the user accross the website, this website uses its own cookies and from third parties, like Google Analytics and other similar activity tracking software. Read the Privacy Policy
33HOPS, IT Consultants Download XSIBackup
33HOPS ::: Proveedores de Soluciones Informáticas :: Madrid :+34 91 930 98 66Avda. Castilla la Mancha, 95 - local posterior - 28700 S.S. de los Reyes - MADRID33HOPS, Sistemas de Informacion y Redes, S.L.Info

©XSIBackup Extended Rotation

Advanced rotation features to build standalone data protection solutions

(*) Do not forget to start at Rotation documentation in the ©XSIBackup-DC manual.

Before you consider anything else

©XSIBackup-DC's rotation features are heavily dependent on the date time stamp

$(date +%Y%m%d%H%M%S)


It is simply a typical timestamp containing the date and time in a pseudo ISO time format (without any separator characters), namely: YYYDDMMhhmmss. If you are a sysadmin you should be used to this kind of naming scheme and if you are an advanced user, it's about time that you do ;-)

The code you are seeing above is just the bash date command being used to generate that string. You will normally use that string, and not other, to automatically generate subfolders where to store backups or replicas. It is indeed the format that the deduplication engine uses to generate backup subfolders.

(*) If you overlook the above and act as if this constraint didn't exist, you won't get rotation working properly.

Rotate VM backup sets

Different ways to rotate your backups & replicas

There are different approaches to rotate your data, you should choose the option that best fits your needs.

--rotate=N (post): rotate a fixed number of folders (N). Easy to configure if you have space enough to not have to worry about it.

--rotate=Nd (post): rotate a fixed number of days (N). Once some backup folder becomes older than N days, it will be automatically deleted.

--rotate=NGB/NTB (pre): backup up until a fixed predefined amount of space (N) has been reached and then start deleting the eldest folders to make room.

--rotate=max (pre): this flag will start to delete the eldest folders in some volume once the free space is below 110% of the VM about to be backed up.

(*) Post & pre means wether the pruning process is performed at the end of the backup job or right before backing up each VM.

Differences in rotating --backup or --replica sets

Backup sets are generated using the above timestamp, when you add the --rotate argument, whether with a single figure to rotate by number of backups, by volume of data or by days, the rotation process will know where to perform that rotation. It will somehow be obvious, as the backup repositories are already generated with a defined structure that allows to easily know where each thing is.

When you use the --replica action to generate directly usable copies of VMs, you are allowed to place this replicas at any point in a local or remote file system. Thus, it will not be that obvious to the rotation process where it has to count accumulated data to know whether it was exceeded in order to delete old folders. It uses some guesses though, which are useful and work well most of the times.

It checks to see if the last dir where the replica is being placed is using the above time stamp convention. If so, it will use the directory immediately below as the root of the rotation. Thus the older time stamped folders in that directory will be deleted by the rotation process and the volume of data accumulated in that root dir will be used to compare against the predefined limits.

--rotate-at

This is yet another argument associated to the rotation process. This is the root folder where ©XSIBackup-DC will consider contents to be stored at. Thus, when you use some user defined space, i.e.: --rotate=500GB, it will check the size of the contents of this folder.

You can use the --rotate-at argument, to which you can pass any local or remote path. When any of the predefined limits are reached, the rotation process will delete the eldest folders in that root path.

When a timestamped folder in the form YYYYMMDDhhmmss is detected at the end of the target argument ©XSIBackup-DC will automatically consider the previous directory to be the root of the rotation, nonetheless you can tweak that to your needs.

When does pruning happen when you employ --rotate

There are two different behaviours: when you rotate per number or per days, the pruning process is performed at the end of the backup, whereas when you rotate by volume of data the pruning is performed right before each VM backup, should some space need to be freed.

Whatch out before you go

Size rotation feature requires as much free space as 110% of the full VM nominal size, namely 10% more space than the nominal size of the VM. It doesn't matter if the disk is sparse and only a part of the space is used. Thus, if your VM has two 100GB disks with only half of the space used, you will need 220GB free to backup that VM. It won't use 220GB though, just 100GB (the non-zero data). This implies that in the end you have to take this fact into account when reserving space.

Should you want to create a set of replicas for some VM which is 200GB in size, you will need at least 440GB to store them, otherwise the rotation process will delete all previous replicas every time. In the end you will be able to keep 2 replicas in those 440GB. If you enlarge the backup user space to 540GB (--rotate=540GB) you will be able to keep 3 replicas, but you will never use more than 300GB of real space, even though your set limit will be 540GB.

This applies of course to sparse files, which is the default virtual disk format that ©XSIBackup-DC creates. Should you use some volume that doesn't allow sparse files, you would need to consider the full VM size as non-zero data, it's weird nowadays though, most of the FSs you may use are indeed able to handle sparse files.

What it means in the end is that you will always need to leave a margin equal to the full size of the VM plus an additional 10% and that used space calculation will be made without taking compression into account. Our space calculations when rotating based on space figures are conservative to the extreme on purpose. If you have a backup volume and you want to fill it up, good luck (you will need it), try with a fixed rotation strategy or with a fixed number of days, but don't complain if you end up hurting yourself.

This applies as well to backup repositories, although in this case the size of the backup will be drastically reduced due to deduplication plus eventual compression (default) on the backup repository. First time you backup a VM to a repository, you will need roughly speaking 50-60% of its non-zero data volume to store it, still you will be asked to provide 110% of its nominal size. On subsequent backups just the differential data volume will be required, still you will need 110% of the VM nominal size to be available.

Note that this space requirement is applied on a per VM basis. Thus, you don't need 110% of the whole volume of all the virtual disks in a Virtual Machine set to be able to back it up, just 110% of the biggest one at the time to back it up.

The eventual need for space is recalculated previous to backing up each new virtual machine. Only if at that moment some lack of space is detected, will the rotate process be triggered to make it. Nonetheless the deletion of objects is performed at a timestamped folder level and that timestamped folder may contain multiple virtual machines.

You need a margin equal in size to 110% of your VM nominal size. If you can't afford to have that margin then use some other kind of rotation.
This page was last modified on 2021-07-17



Website Map
Resources & help
33HOPS Forum
Index of Docs

©33HOPS site relies on the following technologies and partners:
SSL Protocol PayPal Payment Gateway Stripe Payment Gateway

©33HOPS Sistemas de Información y Redes, S.L. | VAT No: ESB83583716 | Avda. Castilla la Mancha, 95, local posterior, 28701 San Sebastián e los Reyes (Madrid) Spain



Fill in to download
The download link will be sent to your e-mail.
Name
Lastname
E-mail


            Read our Privacy Policy

(*) DC & Pro users, please login to your user area to download