Last updated on Wednesday 28th of June 2023 10:54:55 AM

©XSIBackup-App, add local storage

Backup ©VMWare ©ESXi virtual machines to local storage .vmdk or RAW disks

What is ©XSIBackup-App It is an appliance based in Linux CentOS 7 Operating System including ©XSIBackup 2.0 and an nCurses GUI. It is able to backup VMs stored in any ©VMWare ©ESXi host 5.5 or above reachable over SSH protocol. You can download it from Sourceforge.

So you downloaded the appliance, now what. Well, it offers almost endless possibilities: backup over IP to some other remote server, connect to cloud services, etc...

In this post we are going to cover:

- Local virtual disks. Regular virtual disks mounted in ©XSIBackup-App
- Raw disks or RDM (Raw Device Mapping) disks. Physical disks attached to a VM.

In this post we are going to cover how to add local storage to the appliance to host your backups there. We will consider local storage anything mounted in the appliance's file system: local disks, NFS, CIFS, raw devices.

Although there are other technologies that may appear local to the appliance's file system, like cloud storage, we will leave that for another monographic post.

Local disks

We will consider "local disks" virtual disks attached to the appliance as regular VMDK files, no matter whether thin, thick or eager zeroed

Local disks are better as backup storage than, let's say an NFS volume, as latency is negligible and in the order of microseconds.

Lower latency means faster throughput rates, although when doing things in a good working condition LAN this won't affect your transfer speeds much. In case of accessing servers in the internet with latencies over 10ms you will start to notice the overhead caused by network latency.

To add a local disk to the appliance, just proceed as with any other Linux VM:

1/ Add a virtual disk of the desired capacity. It can be of any type: thin or thick. We'll treat raw disks at the end of this post.

Add a virtual disk to XSIBackup-App Add a virtual disk to ©XSIBackup-App
Please, note that from this point, once the new HD has been added to Appliance VM, you can use the GUI to complete making it visible to its file system, formatting and mounting it. Nonetheless, by doing things in the command line you will have more control and a deeper knowledge on what's going on.

The GUI will always follow the same procedure: create an LVM2 Physical Volume for the disk, a Volume Group on top of the PV and a Logical Volume on top of the VG, then format it and mount it at /mnt/XSI/hds. By doing this tasks in the command line you can achieve whatever configuration you want, no matter how intricate it is.

2/ Detect the disk from within the Appliance's Linux OS (CentOS 7). Once added as virtual hardware you can detect the new disk by rebooting the guest or by running the following code snippet, which will iterate the SCSI adapters and add any newly detected HD.

for n in 0 1 2;do echo "- - -" > /sys/class/scsi_host/host${n}/scan;done

Now list the HDs.

[root@XSIBackup-App ~]# ls -la /dev/sd*
brw-rw---- 1 root disk 8, 0 Jun 15 20:07 /dev/sda
brw-rw---- 1 root disk 8, 1 Jun 15 20:07 /dev/sda1
brw-rw---- 1 root disk 8, 2 Jun 15 20:07 /dev/sda2
brw-rw---- 1 root disk 8, 3 Jun 15 20:07 /dev/sda3
brw-rw---- 1 root disk 8, 3 Jun 15 20:07 /dev/sdb
brw-rw---- 1 root disk 8, 3 Jun 15 20:07 /dev/sdb1
brw-rw---- 1 root disk 8, 3 Jun 15 20:07 /dev/sdc

Our new 1TB disk is /dev/sdc

You can add as many disks as you want. The appliance's system is installed on top of disk /dev/sda. The first disk that you add will normally be /dev/sdb

3/ Format the disk.

Note how in this tutorial we are differing from the default behaviour of the GUI, which is to use LVM2 to create the partitions. In this post we are creating a primary partition on top of a disk and then formatting the partition. This is simpler, but can't be modified once it has been created. On the other side LVM2 allows to easily extend partitions, resize them, add new disks, etc...

There are various tools to format HDs in Linux. The most well known one has been fdisk for many years. It has one drawback: it hasn't been updated to support new larger disks, that's why parted has emerged as the new partitioning tool by default. We won't enter into the details of how they work. Below the parted command to create a partition extending through the entire new disk /dev/sdc and below the command to format it as XFS

[root@XSIBackup-App ~]# parted -a optimal /dev/sdc mkpart primary 0% 100%

This will create /dev/sdc1 partition extending through all the available space at /dev/sdc

4/ Mount the just added disk somewhere in the local FS. Finally we can easily mount the new disk somewhere. The default dir to store the subdirs where you mount HDs is /mnt/XSI/hds. You should respect this convention, otherwise you take the risk to lose track of where things are mounted as you start to add a few.

mount /dev/sdc1 /mnt/XSI/hds/backup_sdc1

This is the default view of the /mnt/XSI dir.

[root@XSIBackup-App ~]# ls -la /mnt/XSI/
total 0
drwxr-xr-x 7 root root 64 Jun 27 13:46 .
drwxr-xr-x. 3 root root 17 Jan 25 20:50 ..
drwxr-xr-x 3 root root 17 Jun 27 13:56 hds
drwxr-xr-x 2 root root 6 Apr 26 12:55 iscsi
drwxr-xr-x 3 root root 18 Jun 26 21:01 nfs
drwxr-xr-x 3 root root 18 Jun 21 23:36 smb
drwxr-xr-x 3 root root 27 Jun 27 16:24 srvs

Now you can check that the disk is mounted in the chosen mount point by running df

[root@XSIBackup-App ~]# df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 988M 0 988M 0% /dev
tmpfs 1000M 0 1000M 0% /dev/shm
tmpfs 1000M 105M 895M 11% /run
tmpfs 1000M 0 1000M 0% /sys/fs/cgroup
/dev/mapper/xsi-root 6.2G 2.6G 3.7G 41% /
/dev/sda1 1014M 137M 878M 14% /boot
root@10.0.0.100:/ 2.4G 910M 1.5G 38% /mnt/XSI/srvs/10.0.0.100
tmpfs 200M 0 200M 0% /run/user/0
root@10.0.0.101:/ 2.6G 656M 2.0G 25% /mnt/XSI/srvs/10.0.0.101
/dev/sdc1 1.0T 14G 1011G 2% /mnt/XSI/hds/backup_sdc1

Now to use the disk as the target for some ©XSIBackup-App backup command

xsibackup --backup "VMs(WS2019)" /mnt/XSI/hds/backup_sdc1/repo01 --backup-host=10.0.0.101

The above command will create a backup repository at /mnt/XSI/hds/backup_sdc1/repo01 if it doesn't exist and copy the backup data there


Raw disks

We already covered how to create an RDM (Raw Disk Mapping) disk in a previous post. An RDM disk is nothing but a physical HD attached to a VM. It is slightly faster than a Virtual Disk and has some advantages like being able to plug the physical disk directly into some USB port at the time to eventually restore in a different machine.

On the other side a Virtual Disk can be a better choice in other circumstances. It's up to you to decide what's best for you. You can format the RAW device in whatever your guest OS allows you to. As always the recommended choice is XFS or ext4, in case XFS is not available for you.

We would not choose exotic options, like NTFS or FAT32, you have been warned. The latter are slower and more error prone, while XFS or etx4 will offer you optimum speed and maximum resiliency.

Choosing some FS such as BTRFS or ZFS is explicitly discouraged. Not because they are bad, but just because their feature sets are not something optimum for the task. We have found that some clients would choose XFS in the belief (wrong) that they would get some advantage from its deduplication features, when ©XSIBackup takes care of the deduplication layer, thus they just achieved a very poor performance in regards to speed and reliability to just achieve a 1-2% additional compression ratio.