Forum ©XSIBackup: ©VMWare ©ESXi Backup Software

That was due to a bug we fixed in earlier 1.5 branch. You can safely delete them if you are using 1.5.1.3. There's nothing more to it than the inconvenience of the place, it was due to a missing root path. Just run

rm -rf /*.map.tmp

Of course you don't need to turn it off first. What would be the point of (c)XSIBackup then?

We though that you were using (c)XSIBackup to backup your VMs. We are a bit puzzled by your question. (c)XSIBackup does not make a backup of the snapshot, it does a backup of the whole VM disks, snapshots are not backups.

Any of the two main actions (c)XSIBackup performs (--replica and --backup) do generate a full copy of the VM unless otherwise especified by excluding some disk. Any algorithm aimed at minimizing the transfer of data does just that: minimize the transferred data or the time it takes to do so. All restore points in a repository contain all the data in your disks at any given point in time.

Thus, using (c)XSIBackup to replicate some -flat.vmdk disk that in turn contains some repository will replicate the disk as it is, fully, bit by bit.

If you still have some doubt on how things work, please do not hesitate to ask, still don't set a procedure up asuming something works in any given way without being 100% sure.

Although when you have big repositories that you want to replicate you should use the method explained above to avoid the intrinsec File System overhead, we have detected some things that don't match in your case, which may in turn point at some resource in your system not being well configured or optimized.

We have carried on some tests in our lab with extremely outdated hardware. This sync was done from a Synology NAS box (DS712+) which is around 10 years old seagate SATA disks to XSIBackup-NAS appliance with a single core on an  i5-4460 CPU @ 3.20GHz test server using Rsync over SSH (huge SSH overhead when compared to raw Rsync).

We like to use old hardware, not only because in a lab with some tenths of servers we can reduce the costs, but also because that forces us to optimize things to the extreme.

time rsync -rlpDv --size-only --progress --partial --whole-file --rsh="ssh" /volume2/backup4/DATACENTER-BACKUP-1M root@a.b.c.d:/home/backup/volume1/repo01

...
...

      812375 100%    1.76MB/s    0:00:00 (total: 1%) (xfer#9697, to-check=194/1079665)
DATACENTER-BACKUP-1M/data/f/f/f/4/3/
DATACENTER-BACKUP-1M/data/f/f/f/4/3/fff435aa59b1e8eb04ecc093b0b0ebec64e11f8a
      718443 100%    1.40MB/s    0:00:00 (total: 1%) (xfer#9698, to-check=179/1079665)
DATACENTER-BACKUP-1M/data/f/f/f/6/9/
DATACENTER-BACKUP-1M/data/f/f/f/6/9/fff69d4bcf6e4684e3b5e31baeaf194df19972c4
      863527 100%    1.47MB/s    0:00:00 (total: 1%) (xfer#9699, to-check=144/1079665)
DATACENTER-BACKUP-1M/data/f/f/f/a/7/
DATACENTER-BACKUP-1M/data/f/f/f/a/7/fffa78ab8160524c411da2464d5ac8feb5438ddf
      416077 100%  696.96kB/s    0:00:00 (total: 1%) (xfer#9700, to-check=90/1079665)
DATACENTER-BACKUP-1M/data/f/f/f/c/e/
DATACENTER-BACKUP-1M/data/f/f/f/c/e/fffce5f0c1297487771c129d4847626d7c02cafc
      803256 100%    1.23MB/s    0:00:00 (total: 1%) (xfer#9701, to-check=51/1079665)

sent 6432649065 bytes  received 712064 bytes  149132.70 bytes/sec
total size is 384855008254  speedup is 59.82

real    718m59.062s
user    4m57.396s
sys     6m45.952s

The total size of the replicated repository is 367.00GB. The first sync took around 36 hours, which is quite a bit for the size of the repo. As each block has to be sent over SSH, there is some overhead due to the SSH encapsulation.

As you can see, the second sync using the --size-only argument, took just 718 mins., In our case we generated a diff data volume of aroud 6.00GB

This are figures that make it worth to turn your head to the alternative solution. Still when using newer hardware with more resources and the raw Rsync protocol, this figures should be drastically reduced.

We will repeat the test with the same hardware, the raw Rsync protocol and publish the results.

There isn't any magical solution, or is there one?...

If you use Rsync (or any other software that works at FS level), it's going to take some time to perform the first sync and also a lot of time to perform the subsequent resyncs.

Using Rsync over SSH adds overhead, thus using the plain Rsync protocol along with the [b]--size-only[/b] argument would be the fastest option at a file system level.

Still, as already commented, this would be the wrong approach if you want to minimize sync time. Just place the repository in a volume and then sync the volume at block level, that's the simplest and fastest way to do it.

Of course you can apply the concept using any available technology that manages volumes at block level, still (c)XSIBackup will come in handy here, as a -flat.vmdk file is a volume that you can indeed sync at block level, thus, the solution to your problem is very straight forward:

Store your (c)XSIBackup repositories in a -flat.vmdk file and then use --replica to replicate that disk anywhere you want. If you use CBT you will achieve in seconds the task that would otherwise take days.

Even though deduplication is more likely to happen on the same VM for obvious reasons, it is quite common in datacenters to deploy VMs from templates, thus, any VM that belongs to the same deployed template will still be highly likely to be sharing a good amount of blocks with some other VMs. In case of big datacenters this will represent a huge amount of data.

It is up to you and your knowledge of your own data to determine and/or test whether you will achieve crossed deduplication between VMs or just serial deduplication among the same set's succesive backups.

You need to put things in context.
t's not the same a repository hosting 1TB than one hosting 50TB.
It's not the same looking for a file in a FS tree hosting 50 million files than splitting the workload.

We still don't know the size of your repository.

When (c)XSIBackup backs data up or restores it, it already knows where each block in the tree goes, thus seek times are negligible. When it looks for it it uses a very fast indexing system, thus the seek time is still very low. Nonetheless, if you generate very big repos and you then want to sync them anywhere else at the file system level, using Rsync per instance. Rsync is going to traverse the whole FS to account for the files to transfer.

Even if you just compare file size + timestamp, you are going to use a big amount of RAM and CPU time just to hold that data, let's say 50M files.

[url]https://serverfault.com/questions/365103/how-to-speed-up-rsync-for-small-files/365124[/url]
[url]https://serverfault.com/questions/914214/rsync-huge-dataset-of-small-files-5tb-m-small-files[/url]

You have different ways to work that around:

- Split the load into smaller sets. (c)XSIBackup generates a tree by the first 5 hexadecimal characters of a block which gives more than a million possible subdirs, i.e. /data/0/d/b/a/f/0dbaf3eecbe81de93ddd55bc46f8263bde615613.
You can easily script something that makes Rsync iterate the first 16 subdirs or the first 16x16, taking in consideration the second level. That way you will minimize the need of RAM and CPU to synchronize the whole set in multiple passes.

- Sync the data via a block device, that way it will be agnostic of the number of files being sync'ed. To do that you would need to sync the volume where you store your data. You have different methods to achieve that: some distributed FS such as Ceph or Gluster FS, or maybe even some much simpler approach, such as LVM RAID or mdadm.

You can't use these methods to create a distributed cluster, as nothing is managing concurrency and thus you would end up with a corrupt FS, still you should be able to use this approach if your remote end mirrored RAID1 half is mounted read only. You would also need to find a way to make RAID1 syncs be async to avoid lags if using it over a network.

[url]https://blog.programster.org/create-raid-with-lvm[/url]
[url]https://www.digitalocean.com/community/tutorials/how-to-create-raid-arrays-with-mdadm-on-ubuntu-16-04[/url]

Native file systems such as ext4 and XFS are among the fastest database alike systems that you can find. You issue is not with the FS speed but with the need to store its info in RAM and operate with the file tree. Any of the above solutions will pay off with a minimum investment in terms of technology and time.

You could use a bigger block size (10, 20 or 50MB). Still, CBT is not compatible with other block sizes than 1MB and incrementing the block size to try to overcome some kind of limitation from part of the backup device is not the best solution.

The most adequate cheap backup device for (c)XSIBackup is some old PC with a big HD and an NVMe/ SSD as write cache with some CentOS 7 distro installed or any other Linux distro with FUSE3 and a fast file system: ext4/XFS

File Systems also play a crucial role. VMFS6 should never be used to store deduplicated backups, your best bet is XFS or ext4. BTRFS is not the best match either. On the other side using some user space FUSE derived FS, like: ZFS, s3fs, etc... to store deduplicated backups is just not feasible.

In any case, we use Synology and works well for us. If you just add data to a repository without ever rotating it you are just going to hit the limits of it's CPU and RAM. Another option is that you detect what is the amount of data that your Synology device can handle and create a new repository when you reach that limit.

We fixed that issue in 1.5.1.3, it is already available to download.

CBT feature is not available to Free version. A checksum based incremental backup is dome instead, which takes much longer, specially on bigger VMs.

An OS that is switched on always generates changes, logs per instance. Just a changed bit will trigger the copy of that whole block (1MB by default).

Yes, the --options=R flag was thought to play along with CBT feature, it will not work with regular replicas.
We probably should extend this feature and will do so in the future.

Why don't you use Rsync with a size + timestamp check, instead of a full checksum comparison?.

(*) To other readers: please note that when [b]wimg[/b] says snapshot, he is really referring to the repositories created by (c)XSIBackup. We thought it was worth clarifying.

You are right, it's a new feature and it still doesn't support that scenario. Still you should be able to solve it by enabling CBT for the disk to then skip it.

Your problem is that the CBT routine will check that every disk has CBT enabled, even if it's an independent disk or raw mapping device. If you enable CBT for it you will pass that check and if on top of that you exclude it, then the disk will be skipped.

UPDATE:

Even if you enable CBT for all disks and you don't explicitly exclude the independent disk, the backup routine will just skip it for being innaccesible. The only drawback would be to have to manually exclude it from the .vmx file when restoring.

Regards